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Abstract 


Contracting  for  services  in  the  Department  of  Defense  (DoD)  has  grown  over 
the  last  21  years.  This  growth  in  dollars  spent  has  brought  increased  political 
attention  and  scrutiny.  The  DoD  has  responded  to  problems  such  as  contract 
mismanagement  and  ill-defined  requirements  by  improving  service  acquisitions,  but 
it  still  has  problems.  The  problems  could  be  from  a  lack  of  standard  definition  for 
success.  Because  contract  success  and  failure  is  recorded  through  the  Contract 
Past  Performance  Assessment  Reporting  System  (CPARS),  this  information  is  used 
for  the  proxy  definition  for  success.  This  definition  was  used  to  address  the  following 
questions:  (1 )  Do  the  types  of  services  being  acquired  affect  the  success  of  a 
service  contract?  (2)  Do  the  contractual  amounts  affect  the  success  of  a  service 
contract?  (3)  Does  the  level  of  competition  used  affect  the  success  of  a  service 
contract?  and  (4)  Does  the  contract  type  affect  the  success  of  a  services  contract? 
This  report  examined  715  CPARS  entries.  The  findings  revealed  that  contractual 
amounts  and  level  of  competition  affect  the  success  of  a  service  contract.  The 
findings  also  revealed  that  the  failure  rate  in  CPARS  is  lower  than  expected.  From 
these  findings,  the  report  presents  a  discussion  of  the  results  and  managerial 
implications,  and  recommends  an  alternate  method  in  completing  CPARS  data. 
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Reporting  System  (CPARS),  Stakeholder  Theory,  Service  Contract  Metrics  of 
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Government-Furnished  Property 

Inspector  General 

Independent  Government  Estimate 

Information  Technology 

Lowest  Price  Technically  Acceptable 

Mission  Installation  Contracting  Command 

National  Defense  Authorization  Act 

Office  of  Federal  Procurement  Policy 

Program  Manager 

Past  Performance  Information 
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PPIRS 

Past  Performance  Information  Retrieval  System 

PSC 

Product  Service  Codes 

PWS 

Performance  Work  Statement 

QAE 

Quality  Assurance  Evaluator 

QASP 

Quality  Assurance  Surveillance  Plan 

R&D 

Research  and  Development 

RFP 

Request  for  Proposal 

SOW 

Statement  of  Work 

SSE  Source  Selection  Evaluator 

USD(AT&L)  Under  Secretary  of  Defense  for  Acquistion,  Technology,  and  Logistics 
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I.  INTRODUCTION 


A.  BACKGROUND 

Historically,  the  Department  of  Defense  (DoD)  has  been  the  single  largest 
consumer  of  services  among  government  agencies  (Government  Accountability 
Office  [GAO],  2003).  In  1990,  the  Army’s  spending  alone  on  services  was 
$13  billion  and  grew  to  $74  billion  by  2010  (Ellman,  Livergood,  Morrow,  &  Sanders, 
2011).  From  2000  to  2007,  contract-for-service  spending  remained  close  to  a  level 
percentage  of  total  DoD  spending  until  taking  a  small  dip  in  2008  (Ellman  et  al., 

201 1).  Over  the  past  21  years,  contracting-for-service  spending  has  grown  in 
relation  to  DoD  spending  on  contracting  for  products,  and  was  the  fastest  growing 
area  for  DoD  contract  spending  at  a  growth  rate  of  6.1%  (Ellman  et  al.,  2011;  see 
Figure  1). 


In  total  spending  terms,  from  1990  to  2010,  with  the  exception  of  2008,  the 
Army’s  spending  on  contracts  for  services  has  exceeded  the  amount  of  spending  on 
contracting  for  products  (Ellman  et  al.,  2011).  Figure  2  shows  how  Army  contracting 
for  services  has  increased  at  a  much  higher  rate  than  Army  contracting  for  products 
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during  this  period.  After  September  1 1 ,  2001 ,  the  Army’s  total  contract  spending 
increased  at  the  fastest  rate  it  had  been  in  21  years  (Ellman  et  al.,  201 1 ).  Spending 
grew  at  139%  for  services,  products,  and  research  and  development  (R&D;  Ellman 
et  al.,  201 1 ).  After  2008,  contracting  for  products  decreased  significantly  while 
contracting  for  services  has  increased.  This  growth  in  size  and  scope  of  the  DoD’s 
expenditures  on  services  has  resulted  in  increased  political  attention  in  the  last  few 
years.  This  increased  political  attention  has  allowed  contracting  for  services  to  be 
highlighted  and  scrutinized. 


•90  '91  ‘92  ^93  '94  '95  '96  '97  '98  '99  00  '01  '02  '03  '04  '05  '06  '07  '08  '09  '10 

Fiscal  Year 

□  Army  Products  (5.7%  21-year  CAGR)  O  Army  Services  (9.1%  21-yearCAGR)  □  Army  R&0(3.4%  21-year  CAGR) 

Figure  2.  Army  Spending  on  Products,  Services,  and  R&D 

(Ellman  etal.,  2011) 

Most  analysts  argue  that  contracting  for  services  is  more  complex  than 
contracting  for  products  or  goods  (Church  &  Schwartz,  2013).  This  difference  in 
complexity  is  because  of  vast  differences  between  the  two  forms  of  contracting. 
According  to  Federal  Acquisition  Regulation  (FAR)  Part  37  (2013),  a  contract  for  a 
service  is  a  contract  in  which  the  contractor  performs  a  recognizable  task  rather  than 
furnishing  an  end  item  of  supply.  Apte,  Apte,  and  Rendon  (2010)  highlighted  how 
services  and  products  contain  different  characteristics.  These  characteristics  include 
“intangibility  of  service  output,  co-production,  simultaneity  of  production  and 
consumption,  the  inability  to  store  services,  and  the  complexity  in  the  definition  and 
measurement  of  services”  (Apte  et  al.,  2010,  p.  4).  Chairman  of  the  Defense  Science 
Board  Jacques  Gansler  published  a  memorandum  identifying  five  findings  in  the 
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board’s  assessment  of  service  acquisitions.  One  of  the  findings  was  that  product 
acquisition  is  fundamentally  different  from  the  acquisition  of  services.  Gansler  (2011) 
stated  that  most  current  regulations,  policies,  training,  standards,  education,  and 
management  structures  are  set  up  to  optimize  product  acquisition,  and  only  a  few 
focused  on  the  optimization  of  service  acquisition. 

These  differences  have  created  problems  with  contracting  for  services.  The 
DoD  Office  of  Inspector  General  has  stated  that  the  DoD’s  contracting  for  services  is 
not  managed  effectively  and  that  the  “current  process  is  decentralized,  insufficiently 
rigorous,  and  unreliable”  (GAO,  2003,  p.  1).  In  addition,  the  DoD  Inspector  General 
(IG)  has  issued  142  reports  during  fiscal  year  (FY)  2003  to  2008  that  relate  to  the 
acquisition  and  contract  administration  process  (Burton,  2009).  These  reports  listed 
deficiencies  that  were  grouped  into  12  categories  ranging  from  completeness  of 
acquisition  support  data  to  material  internal  control  weaknesses  (Berkheimer  et  al., 
2008).  In  testimony  before  the  Subcommittee  on  Technology  and  Procurement 
Policy  in  2001 ,  David  Cooper,  GAO  director  of  acquisition  and  sourcing 
management,  identified  several  problems  with  contracting  for  services  ( Contract 
Management,  2001).  One  of  the  problems  was  that  contracting  officials  were  not 
using  prior  experience  of  services  acquisitions  to  help  define  requirements  more 
clearly.  This  led  to  the  use  of  cost  reimbursement  contracts,  which  placed  the 
government  at  risk  for  cost  overruns,  “despite  39  years  of  past  experience 
purchasing  the  same  services”  ( Contract  Management,  2001 ,  p.  6). 

In  2006,  David  M.  Walker,  serving  at  the  time  as  comptroller  general  of  the 
United  States,  testified  before  the  Subcommittee  on  Appropriations  (DoD 
Acquisitions,  2006).  Walker  testified  that  one  of  the  problems  identified  by  the  GAO 
with  services  acquisition  was  oversight  of  contracts.  He  testified  that  out  of  90 
service  contracts  reviewed  by  the  GAO  to  investigate  acquisition  and  contracting 
issues,  nearly  a  third  had  insufficient  contract  oversight.  His  testimony  highlighted 
the  idea  that  the  contracting  office’s  priority  is  to  award  contracts  and  not  to  ensure 
that  personnel  are  properly  trained  to  conduct  oversight  duties  (DoD  Acquisitions, 
2006).  Without  the  proper  number  of  trained  personnel  for  contract  oversight  and  the 
use  of  poor  contract  arrangements,  the  government  runs  the  risk  of  not  getting  the 
services  it  requests  in  a  timely  manner  and  of  paying  more  than  is  necessary  (GAO, 
2013b).  In  his  memorandum,  Gansler  (2011)  also  identified  this  weakness  in  his 
findings.  He  stated  that  the  workforce  needs  additional  training,  guidance,  and 
experience,  and  that  it  is  inadequately  prepared  to  acquire  and  execute  service 
acquisitions  (Gansler,  201 1).  Because  of  these  problems,  contract  management  has 
been  listed  on  GAO’s  High  Risk  Series  since  1992  and  has  allowed  142  reports  from 
the  DoD  IG  on  acquisition  and  contract  administration  deficiencies  in  five  years 
(GAO,  2013b;  Berkheimer  et  al.,  2008). 
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In  response  to  some  of  the  problems  identified,  the  DoD  has  made  advances 
to  improve  service  acquisition.  Some  of  these  changes  aligned  with  Gansler’s  (2011) 
recommendations,  and  they  are  enforced  by  the  Better  Buying  Power  (BBP)  2.0 
memorandum  from  the  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and 
Logistics  (USD[AT&L])  Frank  Kendall.  An  example  of  this  included  the  designation  of 
a  senior  manager  for  services  acquisition  in  each  DoD  component  (GAO,  2013b). 
BBP  2.0  also  mandated  that  each  component  adopt  a  service  market  segmentation. 
This  forced  every  component  in  DoD  to  adopt  a  standardized  approach  for 
categorizing  spending  on  services  to  assist  with  informed  decision-making  (GAO, 
2013b). 

B.  PROBLEM  STATEMENT 

Even  with  the  advancements  in  service  acquisitions,  there  are  still  problems 
with  the  way  service  contracts  are  managed.  These  problems  could  be  related  to  the 
fact  that  there  is  not  an  agreed-upon  definition  of  a  successful  service  contract 
across  the  DoD  (Hagan,  Spede,  &  Sutton,  2012).  The  lack  of  an  agreed-upon 
definition  of  success  has  made  it  difficult  to  measure  a  service  contract  and  has 
made  managing  and  improving  services  aquisition  increasingly  difficult. 

C.  PURPOSE 

The  purpose  of  this  research  is  to  identify  variables  in  the  service  contracting 
process  that  promote  successful  service  contracts.  The  objective  of  this  research  is 
to  build  on  the  understanding  and  knowledge  that  has  been  generated  in  past 
research  and  use  this  knowledge  to  advance  the  acquisition  of  service  contracts. 

This  research  examines  how  specific  variables  affect  the  success  of  a  service 
contract  in  order  to  improve  services  acquisition  management  in  the  DoD. 

D.  RESEARCH  QUESTIONS 

The  primary  question  addressed  in  this  research  is,  what  variables  drive 
successful  service  contracts?  This  research  focuses  on  four  research  questions  that 
explore  the  variables  that  result  in  successful  service  contracts.  The  variables 
selected  to  answer  the  research  questions  are  the  types  of  service,  contractual 
amounts,  level  of  competition,  and  contract  type.  This  research  focuses  on 
answering  four  specific  research  questions: 

1 .  Do  the  types  of  services  being  acquired  affect  the  success  of  a  service 
contract? 

2.  Do  the  contractual  amounts  affect  the  success  of  a  service  contract? 

3.  Does  the  level  of  competition  used  affect  the  success  of  a  service 
contract? 
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4.  Does  the  contract  type  affect  the  success  of  a  services  contract? 

The  answers  to  these  questions  determine  whether  a  relationship  exists 
between  these  four  service  contract  variables  and  the  success  of  the  contracts.  This 
information  will  prove  valuable  in  identifying  variables  that  ensure  a  successful 
service  contract.  The  next  section  discusses  the  methodology  of  this  research. 

E.  METHODOLOGY 

This  research  was  performed  by  conducting  a  literature  review  of  the  service 
contract  management  process,  the  contractor  performance  assessment  reporting 
system  (CPARS),  stakeholder  theory,  and  previous  studies  of  service  acquisition 
management.  The  DoD  CPARS  database  was  accessed  to  obtain  data  on  service 
contracts  for  specific  service  types.  Statistical  analysis  was  conducted  on  the 
CPARS  data  to  draw  conclusions  on  how  certain  variables  affect  the  success  of 
service  contracts.  The  next  section  discusses  the  benefits  and  limitations  of  this 
research. 


F.  BENEFITS  AND  LIMITATIONS 

The  purpose  of  this  research  is  to  identify  variables  in  the  service  contracting 
process  that  promote  successful  service  contracts.  The  benefits  of  this  research 
include  building  on  knowledge  gained  from  previous  research  on  service  contracting. 
It  is  the  first  empirical  study  linking  contracting  success  to  variables  involved  in 
contracting  for  services.  This  research  gives  empirical  data  showing  how  these 
variables  affect  service  contracting.  The  research  identifies  variables  that  relate  to 
the  success  of  service  contracting. 

Although  the  purpose  of  this  research  is  to  identify  variables  in  the  service 
contracting  process  that  promote  successful  service  contracts,  it  is  not  without  its 
limitations.  The  research  limitations  include  the  analysis  of  contracts  only  selected 
from  the  Army,  contracts  only  submitted  by  Mission  Installation  Contracting 
Commands  (MICCs),  and  contracts  from  only  five  MICC  offices.  The  second 
limitation  is  that  only  contracts  that  relate  to  four  selected  service  types  were  used 
for  the  research.  The  final  limitation  is  the  use  of  CPARS  for  a  proxy  definition  of 
success  and  problems  that  have  been  identified  with  the  CPARS  process. 

The  first  limitation  is  that  the  use  of  only  Army  contracts,  only  MICC  contracts,  and 
only  contracts  from  five  MICC  offices  were  chosen  for  the  contracts  to  be  pulled  from 
and  analyzed.  This  limitation  leaves  a  large  area  of  contracting  from  which  data  was 
not  gathered  and  analyzed. 


The  second  limitation  is  that  only  contracts  related  to  four  selected  service 
types  were  chosen  to  be  analyzed.  These  service  types  were  R,  J,  S,  and  D.  R-type 
contracts  are  contracts  for  professional,  administrative,  and  management  support 
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services.  J-type  contracts  contain  maintenance,  repair,  and  rebuilding  of  equipment 
type  services.  S-type  contracts  are  contracts  for  utilities  and  housekeeping  services. 
D-type  services  are  for  automatic  data  processing  and  telecommunications  services 
(Apte,  Apte,  &  Rendon,  2012).  This  limitation  leaves  some  services  from  which  data 
was  not  gathered  and  analyzed. 

The  final  limitation  in  this  research  is  the  use  of  the  CPARS  assessment  as  a 
proxy  for  contract  success,  and  the  problems  that  have  been  identified  with  the 
CPARS  process.  The  use  of  the  CPARS  assessment  as  a  proxy  for  contracting 
success  allows  for  stakeholders  to  apply  their  definition  of  success  to  their  contract. 
This  definition  may  be  in  conflict  with  another  stakeholder  involved  in  the  contracting 
process.  This  area  is  covered  in  more  detail  in  the  literature  review  in  Chapter  II.  The 
final  part  of  this  limitation  is  the  problems  that  have  been  identified  with  CPARS.  This 
limitation  includes  agencies  not  reporting  into  the  database  at  all  or  not  reporting  in  a 
timely  manner.  It  also  includes  the  limitations  of  agencies  reporting  contract  past 
performance  inaccurately  into  the  federal  database.  This  limitation  is  also  discussed 
more  thoroughly  in  Chapter  II. 

G.  SCOPE  AND  ORGANIZATION  OF  RESEARCH 

This  report  is  organized  into  five  chapters.  Chapter  I  contains  the  background 
information,  the  problem  statement,  the  purpose  of  this  research,  research 
questions,  research  methodology,  benefits  and  limitations  of  the  report,  and  the 
scope  and  organization  of  this  research.  Chapter  II  contains  the  literature  review  of 
the  service  contract  management  process,  the  CPARS  system,  stakeholder  theory, 
issues  regarding  CPARS,  and  previous  studies  that  have  been  conducted  on  service 
contract  management.  Chapter  III  provides  a  methodology  of  how  the  research  was 
conducted.  It  describes  how  the  past  performance  information  was  gathered, 
categorized,  and  analyzed.  Chapter  IV  reveals  the  results  of  the  analysis  and  the 
implications  for  DoD  service  acquisition.  Finally,  Chapter  V  provides  a  summary  and 
conclusions,  and  addresses  areas  for  further  research. 

H.  SUMMARY 

This  chapter  discussed  the  background  information,  the  problem  statement, 
the  purpose  of  this  research,  research  questions,  research  methodology,  benefits 
and  limitations  of  the  report,  and  the  scope  and  organization  of  this  research. 

Chapter  II  contains  the  literature  review  of  the  service  contract  management 
process,  the  CPARS  system,  Stakeholder  Theory,  issues  regarding  CPARS,  and 
previous  studies  that  have  been  conducted  on  service  contract  management. 
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II.  LITERATURE  REVIEW 


A.  INTRODUCTION 

The  purpose  of  this  chapter  is  to  provide  an  overview  of  the  DoD’s  service 
acquisition  process  and  the  current  structure  for  acquiring  these  services.  It  begins 
with  an  examination  of  the  current  service  acquisition  process  and  the 
responsibilities  of  the  stakeholders  involved.  Additionally,  this  chapter  provides  a 
discussion  of  how  a  contractor’s  past  performance  is  recorded  in  the  Contractor 
Performance  Assessment  Reporting  System  (CPARS)  for  use  in  future  source 
selections  and  the  problems  that  have  been  identified  with  the  process.  It  then 
discusses  how  success  may  differ  between  each  stakeholder  and  the  impacts  that 
these  differences  have  on  service  contracts.  Finally,  the  chapter  concludes  with  a 
discussion  of  related  research  that  has  built  a  foundation  for  this  project  with  a  look 
at  six  previous  studies  on  services  acquisition  management  in  the  DoD. 

B.  SERVICE  CONTRACT  MANAGEMENT  PROCESS 

The  service  contract  management  process  is  distinctly  different  from  the 
product  contract  management  process.  Apte  et  al.  (2010)  highlighted  how  services 
and  products  contain  different  characteristics.  These  characteristics  include 
“intangibility  of  service  output,  co-production,  simultaneity  of  production  and 
consumption,  the  inability  to  store  services,  and  the  complexity  in  the  definition  and 
measurement  of  services”  (Apte  et  al.,  2010,  p.  4).  Because  of  these  unique 
characteristics,  services  acquisition  must  follow  a  unique  process. 

The  service  acquisition  process  begins  with  the  identification  of  a  service 
requirement  that  is  essential  for  the  organization  to  fulfill  its  mission  (DoD,  2012). 
The  acquisition  process  includes  a  planning  phase,  development  phase,  and 
execution  phase.  The  process  results  in  a  service  that  is  performed  in  a  prescribed 
amount  of  time  and  that  fulfills  the  mission  requirements  (DoD,  2012).  The  service 
acquisition  process  consists  of  seven  steps,  which  are  subtasks  to  the  three  phases 
introduced  previously  and  shown  in  Figure  3:  plan,  develop,  and  execute  (DoD, 
2012).  Steps  1-3  occur  during  the  planning  phase.  Steps  4  and  5  take  place  during 
the  development  phase,  and  Steps  6  and  7  take  place  during  the  execution  phase 
(DoD,  2012). 
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Figure  3.  The  Services  Acquisition  Process 

(DoD,  2012) 


The  Planning  Phase 

The  planning  phase  is  the  first  phase  of  the  service  acquisition  process  and 
consists  of  Steps  1, 2,  and  3  (DoD,  2012).  During  this  phase,  the  acquisition  team  is 
formed  and  receives  guidance  from  their  associated  leadership.  The  acquisition 
team  reviews  current  service  contracts  and  the  marketplace  to  create  a  clear  picture 
of  the  specific  needs  the  service  contract  must  fulfill  (DoD,  2012). 

Step  1:  Form  the  Team 

The  acquisition  team  is  formed  during  Step  1  of  the  service  acquisition 
process  (DoD,  2012).  The  purpose  of  this  step  is  to  ensure  that  the  acquisition  team 
is  formed  and  that  the  team  is  provided  the  proper  resources  to  see  the  contract  to 
completion.  This  step  is  further  broken  down  into  six  sub-elements,  as  seen  in 
Figure  4  (DoD,  2012). 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 


These  six  sub-elements  ensure  the  formation  of  the  acquisition  team  (DoD, 
2012).  This  begins  with  agency  leadership  support  in  acquiring  the  required  service. 
The  acquisition  team  is  then  officially  formed  and  assigned  individual  responsibilities 
The  acquisition  team  consists  of  a  contracting  officer,  program  manager,  contracting 
officer  representative,  finance  manager,  and  customer/end  user.  The  following 
paragraphs  explain  the  roles  of  the  acquisition  team  members. 

Contracting  Officer 

The  FAR  defines  a  contracting  officer  as  an  individual  with  responsibility  and 
authority  to  obligate  the  government  and  enter  into,  administer,  terminate,  and 
properly  close  out  contracts  (FAR,  2.101).  Contracting  officers  pursue  the  interest  of 
the  United  States  by  ensuring  the  proper  performance  of  all  contracts  and  enforcing 
their  compliance  with  the  statutory  requirements  of  public  policy.  A  contracting 
officer’s  duties  are  not  limited  to  the  solicitation  and  award  of  a  contract.  The 
contracting  officer  retains  contracting  officer  authority  and  the  associated 
supervisory  responsibilities  throughout  the  life  cycle  of  the  contract  (Apte  &  Rendon, 
2007).  Some  DoD  contracting  officers  are  civilian  professionals  serving  in  1102 
positions.  These  1102  civilian  employees  “include  positions  that  manage,  supervise, 
perform,  or  develop  policies  and  procedures  for  professional  work  involving  the 
procurement  of  supplies,  services,  construction,  or  research  and  development  using 
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formal  advertising  or  negotiation  procedures”  (Office  of  Personnel  Management 
[OPM],  1 983,  p.  3).  A  full  description  of  the  duties  of  the  contracting  officer  can  be 
found  in  FAR  1.602-2. 

Program  Manager 

The  program  manager  is  responsible  for  overseeing  the  acquisition  team  and 
is  primarily  responsible  for  the  cost,  schedule,  and  performance  of  the  government 
procurement.  Program  managers  make  project  decisions  and  lead  project  execution 
through  effective  collaboration.  They  maintain  continuous  communication  through 
integrated  project  processes  (Apte  &  Rendon,  2007).  The  program  manager 
understands  that  the  need  of  the  agency  leads  the  market  research  effort  and 
oversees  the  award,  execution,  and  closeout  of  the  program.  Previous  research 
indicates  that  in  many  service  acquisitions,  a  project  manager  or  project  team  is  not 
assigned  and  the  contracting  officer  assumes  the  responsibilities  of  the  project 
manager  or  project  team  leader  (Apte  &  Rendon,  2007). 

Contracting  Officer  Representative 

The  contracting  officer  representative  (COR)  is  not  a  contracting-related 
position  and  does  not  have  the  same  statutory  responsibilities  as  the  contracting 
officer  and  program  manager.  The  COR  is  a  subject-matter  expert  who  uses 
technical  expertise  to  supervise  the  execution  of  a  contract  and  provides  periodic 
updates  to  the  contracting  officer.  The  assessing  official  representative  (AOR)  acts 
as  the  “eyes  and  ears”  of  the  contracting  officer  and  is  a  vital  part  of  the  acquisition 
team  (FAR,  2013,  Part  2).  The  COR  does  not  hold  a  warrant  and  does  not  have  the 
authority  to  award,  alter,  or  terminate  contracts  on  behalf  of  the  U.S.  government.  A 
detailed  description  of  the  responsibilities  of  the  COR  can  be  found  in  FAR  1 .604. 

Finance  Manager 

The  finance  manager  serves  as  a  financial  advisor  to  the  acquisition  team. 
Financial  managers  work  to  ensure  that  the  acquisition  team  is  operating  within  the 
constraints  of  government  authorizations  and  appropriations  in  order  to  prevent  a 
violation  of  fiscal  law  (DoD,  2012).  This  ensures  that  the  acquisition  team  has 
appropriated  funds  available  for  the  contract  and  complies  with  the  purpose,  time, 
and  amount  for  the  contract  in  accordance  with  statutory  requirements. 

Customer/End  User 

The  customer  serves  as  the  client  to  the  acquisition  team.  The  customer  must 
fully  understand  the  agency’s  needs  and  be  able  to  clearly  convey  those  needs  to 
the  acquisition  team.  This  understanding  allows  the  acquisition  team  the  ability  to 
create  a  cohesive  requirements  document  (DoD,  2012). 
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Each  of  the  service  contract  stakeholders  have  their  individual  definition  of  a 
successful  service  contract,  which  often  puts  them  in  conflict  with  each  other.  This  is 
discussed  further  in  the  stakeholder  theory  section. 

With  responsibilities  allocated,  the  process  begins  to  move  toward 
understanding  the  stakeholders  and  the  results  they  desire.  During  this  part  of  the 
process,  individual  stakeholders’  desires  are  consolidated  to  ensure  that  the  proper 
services  are  acquired.  After  the  stakeholders  are  in  agreement  with  each  other,  the 
acquisition  team  develops  a  plan  for  communicating  with  the  stakeholders  and  the 
frequency  of  that  communication.  The  final  parts  of  this  step  involve  the  creation  of  a 
plan  to  document  the  information  gathered  in  subsequent  steps  and  to  identify 
training  requirement  mandates  for  the  acquisition  team  members  (DoD,  2012). 

Step  2:  Review  Current  Strategy 

Step  2  of  the  service  acquisition  process  is  to  review  the  current  acquisition 
strategy  being  conducted  (DoD,  2012).  This  process  involves  analyzing  the 
requirements  and  desired  results  of  current  service  contracts  and  attempting  to  draw 
parallels  to  the  service  being  acquired.  In  this  step  the  acquisition  team  attempts  to 
gather  all  the  information  necessary  to  develop  a  list  of  the  exact  requirements 
needed  by  the  service.  This  step  contains  10  sub-elements,  as  seen  in  Figure  5 
(DoD,  2012). 
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Step  Two  -  Review  Current  Strategy 


ReviewCurrent  Strategy 
Sqb-element? 

2.1  Identify  current  riitatwes,' con  tracts 

. »  "  i . . . . 

2.2  Review  and  document  current 
performance  (cost  quality,  schedule) 

,  - , - 

2  3  Begin  program  risk  identification 

i  ■ 

2.4  Document  current  processes 

■  ■ 

2.5  Determine  Status  of  GFP/GFM  Facilities 

i  i  ~ 

2.6  Stakeholder  submits  current  and 
projected  requirements  forecast 

i  ■ 

2.7  Review  current,  statutory  requirements 

i  i 

2  8  Define  (at  a  high  level)  desired  results. 

»  i  ~ 

2.9  Review  current  performance  &  desired 
results  with  stakeholders  and  users 

^  -  -w 

2.10  Refine  desired  results  and  validate 
with  stakeholders 

. 'sr . 


W»or 

R«jts 


Uaon 
Rtflu  urn** 


Figure  5.  Review  Current  Strategy 

(DoD,  2012) 


These  10  sub-elements  assist  the  acquisition  team  in  developing  a  thorough 
understanding  of  the  service  requirements  (DoD,  2012).  This  process  begins  with 
identifying  current  service  contracts  that  are  similar  to  the  requirements.  These 
services  are  then  analyzed  to  gain  a  better  understanding  of  the  service 
requirements  (DoD,  2012).  The  acquisition  team  determines  the  risks  that  require 
mitigation  and  documents  the  information  gathered  thus  far  (DoD,  2012).  The 
acquisition  team  also  determines  whether  government-furnished  property  (GFP)  is 
used  in  the  current  acquisition  and  determines  whether  it  is  needed  for  their  service 
(DoD,  2012).  The  final  steps  of  this  process  involve  an  attempt  to  add  a  greater  level 
of  detail  to  the  desired  outcomes  and  to  ensure  the  stakeholders  agree  with  the 
acquisition  team  (DoD,  2012). 

Step  3:  Market  Research 

“Market  research  is  required  by  FAR  Part  10  and”  is  essential  for  the 
successful  implementation  of  a  service  contract  (DoD,  2012,  p.  18).  This  step 
involves  gathering  price  and  performance  information  from  the  marketplace 
associated  with  the  service  (DoD,  2012).  A  detailed  understanding  of  the  market 
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assists  the  acquisition  team  in  acquiring  the  proper  service  and  avoids  wasting 
taxpayers’  dollars.  There  are  seven  sub-elements  in  this  step,  as  shown  in  Figure  6 
(DoD,  2012). 


Step  Three  -  Market  Research 


Service  Acquisition  Process 


Market  Research 
Sub-elements 

_ »  ■ _ 

1  Take  a  team  approach  to  market 
research 

l'"" . . . "«•  •" 

3 2  Determine  data  sources 

»  i 

3.3  Develop  a  standardized 
interviewguide 

•  »  i 

3.4  Conduct  market  research 
(include  both  providers  and 
consumers) 

3.5  Request  information  from  service 
providers  (informal  or  formal) 

I  I 

3.6  Analyze  market  research 

»  » 

3.7  Document  market  research  and 
trends 


Figure  6.  Market  Research 

(DoD,  2012) 


These  seven  sub-elements  ensure  that  the  acquisition  team  has  a  detailed 
understanding  of  current  market  practices  in  the  area  of  the  service  being  acquired 
(DoD,  2012).  The  process  begins  with  dividing  the  information  needed  among  the 
acquisition  team  members  and  determining  where  the  team  can  gather  that 
information.  The  information  gathered  must  be  collected  in  similar  formats  across  the 
acquisition  team  so  that  it  can  be  compiled  and  disseminated  appropriately.  The 
acquisition  team  members  can  begin  to  contact  service  providers  after  determining 
the  information  that  is  needed.  The  gathered  information  is  then  analyzed  and 
disseminated  throughout  the  team.  The  final  step  of  this  process  is  to  document  the 
collected  market  research,  as  dictated  in  the  FAR  (DoD,  2012). 


The  Development  Phase 

Steps  4  and  5 — Requirements  Definition  and  Acquisition  Strategy — occur 
during  the  development  phase  (DoD,  2012).  This  phase  requires  a  complete 
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understanding  of  the  service  objectives  and  how  the  service  will  be  funded  (DoD, 
2012).  During  the  development  phase,  the  acquisition  team  gathers  information  from 
the  industry  in  an  effort  to  streamline  the  acquisition  process  and  avoid  potential 
acquisition  problems.  The  final  part  of  this  process  involves  putting  all  the  gathered 
information  together  to  create  an  acquisition  strategy  that  capitalizes  on  the 
information  gathered  in  an  effort  to  obtain  the  correct  service  for  the  government 
(DoD,  2012). 

Step  4:  Requirements  Definition 

Step  4  in  the  service  acquisition  process  involves  compiling  the  information 
gathered  thus  far  and  developing  a  specific  list  of  all  the  requirements  for  the  service 
contract  (DoD,  2012).  This  step  specifically  involves  taking  a  detailed  look  at  the 
individual  requirements  and  understanding  how  they  interact  with  each  other.  This 
step  is  broken  into  the  eight  sub-elements  shown  in  Figure  7  (DoD,  2012). 


Step  Four  -  Requirement  Definition 


Service  Acquisition  Process 


Requirement  Definition 
Sub-elements 

_ i  ~~i _ 

4.1  Conduct  performance  Risk 

Analysis 

1”  ■ 

4.2  Conduct  a  Requirements  Analysis 

»  ■ 

4.3  Build  Requirements  Roadmap 

■  ■  — 

4.4  Standardize  requirements  where 
posstole  to  leverage  market 
influence 

■  ■  , 

4.5  Develop  a  PWS  or  S00 

1 .  I 

4.6  Develop  the  QASP 

r . —  "Hi 

4.7  Develop  Independent  Government 
Estimate  (IGE)  based  on 
projected  demand  forecast 

■  i 

4.8  Establish  stakeholder  consensus 


Figure  7.  Requirements  Definition 

(DoD,  2012) 

These  eight  sub-elements  facilitate  a  complete  understanding  of  the 
requirements  that  the  service  is  going  to  fulfill,  along  with  the  associated  costs  (DoD, 
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2012).  The  process  begins  with  a  detailed  understanding  of  the  risks  in  acquiring  the 
service.  Most  service  acquisition  projects  are  analyzed  in  terms  of  the  likelihood  of  a 
given  risk  occurring  and  its  associated  severity.  This  information  is  often  compiled  on 
the  DoD  Risk  Analysis  Model  shown  in  Figure  8  (DoD,  2012).  Beginning  with  high- 
risk  areas,  risk  of  contract  failure  or  delay  has  to  be  identified  and  mitigated 
appropriately  (DoD,  2012). 


1.  What  is  the  LIKELIHOOD 
an  event  will  otcur? 


Level 

What  h  the  Ukekhood  the 
Rick  [vent  will  occur? 

a 

Remote 

b 

Unlikely 

c 

Likely 

d 

Hfchfy  Ufcety 

e 

Near  Certainty 

A 


Risk  Analysis  Model  - 
Tying  it  Together 


e 

d 

5  c 
■ 

3  b 


2.  What  is  the  IM 
if  the  event  occurs? 
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IMPACT 
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HIGH  -  Unacceptable.  Major 
R  disruption  likely,  different 
approach  required,  priority 
management  attention  required. 

-  Some  disruption; 
different  approach  may  be 

management  attention 
may  be  needed. 

IOW  -  Minimum  impact; 

G  minimum  oversight  needed  to 
ensure  risk  remains  low. 


Technical  And/  And/  And/  Impact  on 

Performance  or  Schedule  or  Coil  or  Other  teams 

a 

Minimal  or  no  impact 

Minimal  or  no  impact 

Minimal  or 
r>o  impact 

None 

b 

Acceptable  with  tome 
reduction  In  margin 

Additional  resources  required; 
able  to  meet  need  dates 

<s% 

Some  impact 

c 

Acceptable  with  significant 
reduction  In  margin 

Minor  slip  In  key  milestones; 
not  able  to  meet  all  need  dates 

S-7% 

Moderate  impact 

d 

Acceptable;  no  remaining 
margin 

Major  slip  In  key  milestone 
or  critical  path  impacted 

710% 

Major  Impact 

e 

Unacceptable 

Can't  achieve  key  team 
or  major  milestone 

>10% 

Unacceptable 

Figure  8.  Risk  Analysis 

(DoD,  2012) 


Once  the  risks  are  analyzed,  the  acquisition  team  begins  to  create  a 
document  describing  how  to  fulfill  and  prioritize  the  service  requirements  (DoD, 
2012).  One  of  the  tools  used  to  complete  this  task  is  the  requirements  roadmap  tool 
shown  in  Figure  9.  The  requirements  roadmap  clearly  defines  and  prioritizes  the 
service  requirements  through  the  duration  of  the  acquisition  process  (DoD,  2012). 
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The  final  steps  in  this  process  involve  detailing  the  exact  services  required  by 
the  agency  or  customer  and  ensuring  that  the  stakeholders  agree  with  the  estimated 
costs  and  the  prioritization  of  the  requirements  (DoD,  2012). 

Step  5:  Acquisition  Strategy 

The  plan  for  determining  which  type  of  contract  to  use  and  how  the  source 
selection  is  determined  is  part  of  the  acquisition  strategy  (DoD,  2012).  This  step 
involves  putting  together  all  of  the  previously  gathered  information  and  creating  a 
plan  to  achieve  the  contracting  objectives  (DoD,  2012).  This  step  is  broken  down 
into  the  five  sub-elements  seen  in  Figure  10. 
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Figure  10.  Develop  an  Acquisition  Strategy 

(DoD,  2012) 


The  five  sub-elements  of  the  acquisition  strategy  step  begin  with  determining 
the  proper  contract  type  to  use  and  ensuring  that  it  contains  the  correct  incentives  to 
incentivize  the  service  provider  (DoD,  2012).  From  there,  the  acquisition  team 
determines  the  method  that  will  be  used  to  select  the  best  value  source  for  the 
service.  The  acquisition  team  is  assigned  responsibilities  for  evaluating  the 
proposals  received,  and  a  plan  is  created  to  disseminate  the  information  amongst 
the  team.  The  final  step  involves  analyzing  whether  a  draft  request  for  proposal 
(RFP)  should  be  used  in  order  to  obtain  more  feedback  from  the  industry;  this  step  is 
useful  in  reducing  any  misunderstandings  or  confusion  from  the  industry  and  allows 
for  a  smoother  acquisition  process  (DoD,  2012). 

The  Execution  Phase 

All  the  planning  and  development  conducted  thus  far  are  implemented  during 
the  execution  phase  (DoD,  2012).  This  phase  contains  Steps  6  and  7  and  begins 
with  the  publishing  of  a  solicitation  document.  The  solicitation  document  officially 
communicates  requirements  to  the  marketplace.  From  here,  offerors  submit  their 
proposals,  which  are  evaluated  on  their  own  merit,  and  then  each  proposal  is 
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evaluated  for  its  ability  to  meet  the  requirements  listed  in  the  solicitation  document. 
Once  the  contract  is  awarded,  the  contractor  and  the  government  begin  to  operate 
as  a  team  as  opposed  to  individual  entities.  This  phase  could  eventually  lead  to  the 
planning  of  a  follow-on  acquisition  in  the  event  the  service  is  needed  in  perpetuity 
(DoD,  2012). 

Step  6:  Execute  Strategy 

Step  6  marks  the  official  start  to  the  initiation  of  the  service  acquisition,  from 
the  official  request  of  a  service  to  the  selection  of  the  provider  of  that  service  (DoD, 
2012).  As  seen  in  Figure  1 1 ,  this  step  includes  seven  sub-elements. 


Step  Six  -  Execute  the  Strategy 


Execute  Strategy 
Sub-elements 

6.1  Issue  request  for  proposal  (RFP) 
orMPR 

i  i 

6.2  Conduct  source  selection 
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6.3  Pre-Averd  approval  documents 

~i  r 

6.4  Contract  avard 

~l —  I 

6.5  Debrief  unsuccessful  offerors 

HZ  ~T~ 

6.6  Finalize  Quality  Assurance 
Surveillance  Plan 

T  I 

6.7  Post  Award  implementation/ 
transition 


Service  Acquisition 


mss  on 
Rfgurtmert 


Us  von 
Resets 


Figure  1 1 .  Execute  the  Strategy 

(DoD,  2012) 


The  seven  sub-elements  begin  with  the  official  release  of  the  RFP  (DoD, 
2012).  The  proposals  received  from  the  industry  are  evaluated  based  on  the  criteria 
outlined  in  the  acquisition  strategy  from  Step  5.  Once  the  source  selection  has  been 
determined,  the  companies  that  did  not  win  the  bid  are  debriefed.  This  debrief  is 
intended  to  provide  the  companies  with  information  that  could  help  them  be  more 
competitive  on  future  procurements.  The  final  steps  in  this  process  solidify  the 
quality  assurance  surveillance  plan  (QASP;  DoD,  2012). 
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Step  7:  Performance  Management 

Step  7  is  the  final  step  in  the  service  acquisition  process  and  is  essentially  the 
implementation  of  the  awarded  contract  (DoD,  2012).  The  step  involves  actively 
managing  the  contract  to  achieve  the  mission  requirements.  Expectation  and 
contract  management  are  key  components  of  this  step.  This  step  is  broken  down 
into  the  six  sub-elements  shown  in  Figure  12  (DoD,  2012). 


Step  Seven  -  Performance  Management 


ttrltion 

Rnulti 


7.6  Develop  plan  lor  managng 
continuous  mprovement 


Management 
Sub-elements 

I  I 

Transition  to  Performance 
Management 

i  i 

72  Manage  and  admncster  overal 
program 

zrz  ~r~ 

7.3  Manage  Performance  Resute 
~1  I 

74  Conduct  quarterly  suppher  and 
key  stakeholders  performance  reviews 

~~r~  ~r~ 

7.5  Evaluate  effectiveness  of  strategy 

I  I 


Results 


Figure  12.  Performance  Management 

(DoD,  2012) 


The  sub-elements  in  this  step  begin  with  measuring  the  contractor’s 
performance  as  described  in  the  signed  contract  (DoD,  2012).  The  contract  is 
administered  during  this  step  to  ensure  that  all  parties  are  performing  their  assigned 
duties,  which  includes  compensating  the  contractor  for  providing  the  accepted 
services.  Another  important  part  of  contract  administration  is  regularly  informing  the 
contractor  of  his  or  her  positive  or  negative  performance  of  the  services  they  are 
providing.  Analyzing  and  documenting  the  completed  acquisition  concludes  the 
service  acquisition  process  (DoD,  2012).  During  this  final  phase  of  the  service 
acquisition  process,  the  contractor’s  performance  is  documented  for  use  in  future 
source  selections.  The  past  performance  information  is  captured  in  a  database 
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called  the  Contractor  Performance  Assessment  Reporting  System  (CPARS),  which 
is  discussed  in  the  next  section. 

C.  THE  CONTRACTOR  PERFORMANCE  ASSESSMENT  REPORTING 

SYSTEM 

The  FAR  defines  past  performance  as  “an  offeror’s  or  contractor’s 
performance  on  active  and  physically  completed  contracts”  (Part  2).  Past 
Performance  Information  (PPI)  is  “used  to  communicate  contractor  strengths  and 
weaknesses  to  source  selection  officials  and  contracting  officers”  (DoD,  201 1,  p.  50) 
CPARS  is  used  in  contracting  to  capture  past  performance  information  (PPI)  for  use 
in  determining  contractor  responsibility  (FAR,  2013,  Part  9.104),  during  the  source 
selection  process  (FAR,  2013,  Part  15),  and  during  the  contract  administration 
process  (FAR,  2013,  Part  42).  This  research  focuses  on  the  source  selection 
process. 

In  1994,  Congress  viewed  past  performance  to  be  relevant  in  source 
selection  and  included  this  in  the  Federal  Acquisition  Streamlining  Act  (Berkheimer 
et  al. ,  2008).  In  1998,  the  Navy  created  the  CPARS  to  collect  historical  information 
and  to  be  in  compliance  with  the  congressional  act  (Berkheimer  et  al.,  2008).  In 
2004,  the  USD(AT&L),  defense  procurement,  and  acquisition  policy  designated 
CPARS  as  the  “solution  for  collecting  contractor  past  performance  information” 
(Berkheimer  et  al.,  2008,  p.  i). 

The  Purpose  of  CPARS 

CPARS  was  directed  by  the  DoD  and  is  designed  to  ensure  that  up-to-date 
information  is  available  on  a  contractor’s  past  performance,  and  is  intended  to  be 
used  in  the  source  selection  process  (DoD,  2011).  “The  CPAR  assesses  a 
contractor’s  performance,  both  positive  and  negative,  and  provides  a  record  on  a 
given  contract  during  a  specified  period  of  time”  (DoD,  201 1 ,  p.  1 ).  “The  CPARS 
process  is  designed  with  a  series  of  checks-and-balances  to  facilitate  the  objective 
and  consistent  evaluation  of  contractor  performance.  Both  Government  and 
contractor  perspectives  are  captured  on  the  CPAR  form”  (DoD,  2011,  p.  2).  The 
information  contained  in  CPARS  is  designed  to  be  objective,  though  subjective  data 
can  be  used  in  the  event  that  measurement  data  is  not  identifiable  (DoD,  2011). 
Once  CPARS  data  is  entered,  it  goes  into  the  Past  Performance  Information 
Retrieval  System  (PPIRS),  which  collects  past  performance  information  (DoD, 

2012).  It  is  important  to  note  that  the  information  contained  in  PPIRS  “is  a  method  of 
recording  contractor  performance”  and  must  not  be  the  sole  information  used  in 
determining  a  source  selection”  (DoD,  2012,  p.  v).  CPARS  data  entry  is  mandatory 
for  any  service  contract  exceeding  $1  million,  and  the  entries  are  completed  by 
several  different  stakeholders. 
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CPARS  Roles  and  Responsibilities 

Five  individuals  are  responsible  for  the  overall  CPARS  input  process:  the 
activity  CPARS  focal  point,  the  assessing  official  representative  (AOR),  the 
assessing  official  (AO),  the  designated  contractor  representative,  and  the  reviewing 
official  (DoD,  2011).  The  activity  CPARS  focal  point  is  responsible  for  registering  the 
contract  into  the  CPARS  database  and  is  the  individual  who  provides  CPARS 
access  authorization.  The  AOR  has  the  responsibility  of  assisting  the  AO  as  needed 
and  provides  a  narrative  that  contains  information  regarding  the  quality  of  the  service 
(DoD,  2011).  AORs  are  typically  CORs  who  are  often  assigned  from  the  technical, 
functional,  quality  assurance,  specialty,  program  management  or  contracting  offices, 
or  the  end  users  of  a  service.  The  AO  has  the  overall  responsibility  of  ensuring  that 
the  CPARS  data  and  the  overall  service  is  completed.  The  AO  may  be  a  program 
manager  or  the  equivalent  individual  responsible  for  the  management  of  the  service 
acquisition.  The  AO  reviews  the  narrative  submitted  by  the  AOR  and  provides  an 
overall  evaluation  of  the  contractor’s  performance.  If  any  of  the  contractor’s 
comments  are  modified  by  the  AO,  then  the  AO  is  responsible  for  ensuring  that  the 
contractor  is  aware  of  them  prior  to  the  CPARS  process  being  closed  out  (DoD, 
2011). 

The  designated  contractor  representative  is  the  individual  who  receives  the 
completed  evaluations  from  the  AO  (DoD,  2011).  This  representative  then  reviews, 
comments  on,  and  returns  the  evaluation  to  the  AO.  The  designated  contractor 
representative  has  the  ability  to  request  a  reviewing  official  review  if  deemed 
necessary.  The  reviewing  official  has  the  authority  to  provide  a  narrative  to  the 
CPARS  data  and  is  the  individual  responsible  for  signing  the  CPAR,  which  officially 
closes  out  the  CPARS  process.  Once  the  CPAR  is  signed,  the  information  is 
available  through  PPIRS  for  use  by  a  future  source  selection  team  for  a  period  of 
three  years. 

The  quality  of  the  information  in  each  CPARS  data  entry  is  determined  by  the 
quality  of  the  information  provided  by  the  individuals  (DoD,  2011).  Therefore,  the 
effectiveness  of  CPARS  is  determined  by  the  individuals  who  capture  the  data. 
Although  the  CPAR  system  has  the  capacity  to  capture  PPI  and  provide  valuable 
information  to  acquisition  officials,  the  system  requires  timely  and  detailed 
information  from  many  different  stakeholders  to  fulfill  its  potential.  This  often  leads  to 
issues  among  the  stakeholders  involved  in  the  service  acquisition  because  they  all 
may  not  have  the  same  objectives  in  the  way  they  assess  the  contractor’s 
performance.  The  contracting  officer,  program  manager,  COR,  contractor,  finance 
manager,  and  customer  all  have  different  objectives  in  the  way  each  manages  a 
service  contract.  The  contracting  officer  is  primarily  concerned  with  a  contract’s 
adherence  to  statutory  requirements  and  public  policy,  and  while  required  to  support 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 


■21 


cost,  schedule,  and  performance  objectives,  the  program  manager  is  held  solely 
accountable  for  their  management  (FAR,  2.101 ;  DoD,  2012).  The  COR  technical 
expert  would  consider  a  services  contract  successful  if  it  meets  or  exceeds  technical 
requirements  while  not  necessarily  valuing  statutory  requirements  or  the  cost, 
schedule,  and  performance  of  the  contract  (Apte  &  Rendon,  2007).  The  contractor 
values  obtaining  the  highest  profit  possible  while  satisfying  the  needs  of  the 
customer  to  obtain  future  business.  The  financial  manager  is  solely  concerned  with 
ensuring  that  the  acquisition  team  is  operating  within  current  appropriations  and 
authorizations.  Lastly,  the  customer  is  mainly  concerned  with  his  or  her  needs  being 
met  as  expeditiously  and  efficiently  as  possible. 

Based  on  the  different  definitions  of  success  by  the  stakeholders,  it  is  easy  to 
see  how  the  stakeholders  are  sometimes  in  conflict  with  each  other.  The  contracting 
officer  may  need  more  time  to  meet  the  statutory  requirements  of  a  contract,  which 
may  adversely  affect  the  program  manager’s  schedule.  The  financial  manager  may 
not  be  able  to  allocate  enough  money  or  even  the  correct  appropriation  (color  of 
money)  to  a  service  to  meet  the  contractor’s  required  cash  flow.  Lastly,  the  customer 
may  value  a  service  provided  but  the  service  may  not  meet  the  technical 
requirements  according  to  the  COR.  Stakeholder  theory  helps  to  explain  why  these 
different  stakeholders  have  different  objectives  and  how  it  can  assist  in  the 
management  of  a  service  contract. 

D.  STAKEHOLDER  THEORY 

Stakeholder  theory  states  that  in  business,  the  stakeholder’s  responsibility  is 
to  generate  as  much  value  as  possible  for  his  or  her  agency  or  business  (Donaldson 
&  Preston,  1995).  The  founding  father  of  stakeholder  theory,  R.  Edward  Freeman 
(1984),  assumed  that  the  purpose  of  the  corporation  is  to  generate  and  dispense 
some  form  of  wealth  to  various  stakeholders,  and  that  in  order  to  achieve  that 
purpose,  all  of  the  stakeholders  cooperate.  Applying  stakeholder  theory  to  services 
contracting  involves  all  members  of  the  government  acquisition  team,  contractors, 
and  the  end  user  of  a  certain  service.  The  members  of  the  government  acquisition 
team  include  the  contracting  officer,  program  manager,  COR,  finance  manager,  and 
customer  or  end  users.  The  different  stakeholders  represented  in  this  model  have 
very  different  ways  of  measuring  success  that  are  often  in  conflict  with  each  other 
due  to  their  differing  goals.  When  the  government  and  contractor  negotiate  a 
contract,  the  terms  and  conditions  represent  a  compromise  aimed  at  capturing  the 
best  value  for  the  respective  parties.  These  stakeholders  also  input  qualitative  data 
based  on  the  performance  of  the  contractor  into  the  CPARS  database,  which  gives 
them  the  opportunity  to  explain  how  they  measure  the  success  of  a  contract.  Even 
with  different  stakeholders  working  together  to  execute  a  contract,  there  are  still 
issues  with  how  the  process  is  captured  in  the  CPARS  database.  In  addition  to  the 
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different  stakeholders  and  potential  conflicting  stakeholder  goals,  there  are  other 
issues  and  challenges  involved  in  managing  PPI. 

E.  CURRENT  ISSUES  REGARDING  CPARS 

The  DoD  IG  conducted  a  study  in  2008  on  the  controls  and  systems  in  place 
for  contractor  past  performance  records.  They  found  that 

•  39%  were  registered  more  than  a  year  late  from  contract  award; 

•  68%  had  performance  reports  that  were  overdue;  and 

•  82%  of  past  performance  assessment  reports  reviewed  did  not  contain 
detailed,  sufficient  narratives  to  establish  that  ratings  were  credible  and 
justifiable.  (Berkheimer  et  al.,  2008,  p.  i) 


At  the  time  of  this  report,  the  IG  found  that  321  Army  contracts  had  not  been 
registered  in  CPARS  (Berkheimer  et  al.,  2008).  These  contracts  were  not  being 
reported  as  required,  so  there  is  not  any  past  performance  data  in  the  system  on 
these  contracts  (Berkheimer  et  al.,  2008).  The  report  also  found  that  82%  of  the 
performance  assessments  lacked  detailed  narratives  to  support  the  ratings  given  to 
the  contractor  (Berkheimer  et  al.,  2008). 

The  GAO  conducted  another  study  in  2013  in  response  to  two  previous 
studies  in  2009  by  the  GAO  and  a  201 1  study  by  the  Office  of  Federal  Procurement 
Policy  (OFPP;  GAO,  2013a).  In  2009,  the  GAO  found  that  contract  reporting 
systems  were  not  complete,  accurate,  or  (due  to  limitations)  effective  (GAO,  2009). 
The  OFPP  reported  that  the  DoD  lacked  sufficient  narratives  to  justify  ratings  given 
to  the  contractors  (GAO,  2013a).  In  2013,  the  GAO  found  that  PPIRS  compliance  for 
timeliness  was  up  from  56%  in  201 1  to  74%  in  2013.  This  shows  that  a  little  over  a 
quarter  of  the  required  contracts  are  not  submitted  on  time  into  the  reporting  system. 
The  DoD  compliance  was  shown  to  be  just  lower  than  the  average  compliance  of  all 
the  services  at  73%  submitted  in  time  (GAO,  2013a).  DoD  officials  attributed  the 
shortcomings  of  the  reporting  system  to  the  acquisition  workforce  shortages  and 
high  turnovers  (GAO,  2013a). 

The  GAO  and  the  DoD  IG  investigations  showed  that,  although  the  systems 
already  in  place  to  evaluate  past  performance  are  improving,  they  are  still  flawed 
(GAO,  2013a).  The  National  Defense  Authorization  Act  (NDAA)  for  FY2012  requires 
a  DoD  strategy  to  ensure  past  performance  assessments  are  complete,  timely,  and 
accurate  (GAO,  2013a).  These  studies  have  shown  that  the  DoD  still  has 
improvements  to  make  to  be  compliant  with  the  NDAA.  Despite  the  concerns  of 
conflicting  stakeholders  and  the  DoD  IG  and  GAO  findings,  the  use  of  CPAR 
assessments  as  a  surrogate  for  contracting  success  is  based  on  previous  studies  of 
service  acquisition,  which  is  discussed  in  the  following  section. 
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F. 


PREVIOUS  STUDIES 


This  research  is  a  follow-on  to  a  stream  of  past  research  studies  on  services 
acquisition  in  the  DoD  that  started  in  2006.  Apte,  Ferrer,  Lewis,  and  Rendon  (2006) 
observed  that  the  DoD’s  acquisition  volume  increased  in  the  decade  prior  to  2006. 
They  began  an  initial  exploratory  analysis  in  order  to  frame  the  DoD’s  service 
acquisition  environment.  This  research  revealed  several  significant  findings  that 
impact  service  contracting.  They  noted,  as  mentioned  in  the  background  section,  the 
large  increase  in  service  contracting.  They  stated  that  service  contracting  has 
increased  by  66%  since  1999  (Apte  et  al.,  2006).  In  2003,  the  DoD  spent  $118 
billion,  which  is  approximately  57%  of  the  total  DoD  procurement  dollars  (Apte  et  al., 
2006).  They  observed  that  four  services  made  up  approximately  50%  of  total  DoD 
spending  on  services  (Apte  et  al.,  2006).  These  services  were 

•  professional,  administrative,  and  management  support  services; 

•  construction,  repair,  and  maintenance  of  structure  and  facilities; 

•  equipment  maintenance;  and 

•  information  technology  (IT)  services  (Apte  et  al.,  2006). 

Apte  et  al.  (2006)  observed  that  frequent  communication  was  essential  in  the 
success  of  service  contracts  and  discussed  a  new  technique  called  the  Business 
Requirement  Advisory  Group  (BRAG).  When  used  successfully,  the  BRAG  facilitates 
communication  in  the  administration  of  contracts  through  the  contract  life  cycle.  This 
study  brought  attention  to  the  centralization  of  service  contracts.  It  showed  the 
advantages  and  disadvantages  of  centralization,  such  as  the  difficulty  of 
communication  if  the  contract  is  being  executed  at  a  different  location  than  where 
the  contracting  officer  is  located.  According  to  Apte  et  al.  (2006),  the  downsizing  of 
the  acquisition  workforce  and  the  push  to  aggressively  comply  with  Circular  A-76 
(Office  of  Management  and  Budget  [OMB],  2003)  contradict  the  importance  of 
having  the  correct  number  of  trained  acquisition  workforce  to  be  successful.  This 
compliance  with  the  Circular  A-76  causes  the  value  for  the  money  to  not  be  realized. 
The  most  significant  find  in  this  research  is  that  while  the  DoD  has  increased  service 
contract  spending,  the  management  infrastructure  has  not  been  maturing  like 
product  acquisition  (Apte  et  al.,  2006). 

In  2007,  Apte  and  Rendon  completed  their  second  exploratory  research 
project  on  services.  This  research  looked  further  into  the  supply  chain  management 
of  service  contracting  and  the  application  of  a  program  management  infrastructure 
(Apte  &  Rendon,  2007).  Their  research  identified  that  the  traditional  approach  to 
managing  the  service  supply  chain  in  the  DoD  and,  specifically,  the  Air  Force  “does 
not  include  a  centralized  and  coordinated  management”  approach  (Apte  &  Rendon, 
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2007,  p.  33).  The  Air  Force’s  approach  does  not  include  a  “life  cycle,  a  designated 
program  manager,  cross-functional  teams,  or  an  enabling  organizational  structure” 
(Apte  &  Rendon,  2007,  p.  33).  During  their  research,  Apte  and  Rendon  did  observe 
two  types  of  innovative  approaches  to  this  structural  void  to  assist  the  procuring 
agency  in  becoming  more  successful.  The  first  approach  was  the  Air  Education  and 
Training  Center  (AETC).  The  AETC  allows  for  a  coordinated  pre-award  management 
of  services  acquisition.  The  AETC  “does  not  maintain  an  on-site  program  manager” 
(Apte  &  Rendon,  2007,  p.  33).  Not  keeping  a  program  manager  on  site  allows  for  the 
system  to  potentially  fail  because  of  broken  communication  between  the  interested 
parties.  The  Air  Combat  Command  (ACC)  model,  which  uses  the  Acquisition 
Management  and  Integration  Center  (AMIC)  approach  to  service  contracting,  allows 
for  a  unique  process  that  provides  a  cradle-to-grave  acquisition  approach  (Apte  & 
Rendon,  2007). 

In  their  third  exploratory  research  study,  conducted  in  2008,  Apte,  Apte,  and 
Rendon  continued  their  research  into  the  management  of  services  acquisition  at 
military  bases  throughout  the  DoD.  This  research  was  a  survey-based  look  at  the  Air 
Force  and  Navy’s  techniques  for  dealing  with  service  contracts.  This  research 
provided  real  data  on  characteristics  of  service  contracting,  the  management 
approaches  used,  and  the  program  management  issues  inside  the  DoD  (Apte  et  al. , 
2008). 

In  Apte,  Apte,  and  Rendon’s  (2008)  research,  the  Air  Force  covered  four 
service  categories:  professional  management  services,  maintenance,  data 
processing,  and  transportation.  The  researchers  looked  at  the  degree  of  competition, 
contract  type,  and  incentives  used.  The  survey  explored  the  level  of  services  that  are 
acquired  in  the  Air  Force.  The  survey  also  included  questions  asking  whether  a 
project  team  approach  was  used,  who  was  the  leader  of  the  team,  and  who  owns 
the  requirements  for  the  acquisition.  Apte  et  al.  (2008)  found  that  the  Air  Force 
services  they  examined  used  competition  the  majority  of  the  time. 

The  researchers  found  that  the  Air  Force  used  firm  fixed-price  contracts  the 
majority  of  the  time  but  rarely  used  incentives,  as  seen  in  Table  1  (Apte  et  al.,  2008). 
The  data  showed  that  the  majority  of  the  service  acquisition,  throughout  the  life 
cycle,  was  conducted  at  the  installation  level  as  opposed  to  the  regional  level.  The 
research  revealed  that  the  contracting  officer  led  the  project  teams  as  opposed  to 
the  program  manager.  This  indicates  that  there  was  not  program  management 
involvement  in  the  acquisition  process  of  services  at  the  installation  level  (Apte  et  al., 
2008). 
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Table  1.  Contract  Management  Approach  Found  in  the  Air  Force 

(Apte  et  al.,  2008) 
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Apte  et  al.’s  (2008)  research  included  the  following  Navy  services: 
professional  and  administrative,  maintenance  and  repair,  data  processing,  and 
utilities  and  housekeeping  (Apte  et  al.,  2008).  The  survey  for  the  Air  Force  was 
replicated  for  the  Navy  in  this  research.  Apte  et  al.  (2008)  covered  six  regions,  which 
encompassed  66  installations. 

Apte  et  al.  (2008)  found  that  the  Navy’s  organizational-level  handling  of 
service  contracting  was  different  than  the  Air  Force’s.  The  Navy  acquired  a  little  over 
half  of  its  professional  services  at  the  regional  level  (Apte  et  al.,  2008).  Apte  et  al. 
(2008)  found  that  68%  of  the  Navy’s  data  processing  services  were  acquired  and 
managed  at  the  regional  level.  The  researchers  also  found  that  approximately  half  of 
the  utilities  and  housekeeping  services  were  acquired  at  the  regional  level,  and  the 
other  half  were  acquired  and  managed  at  the  installation  level  (see  Table  2;  Apte  et 
al.,  2008). 
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Table  2.  Results  From  Study  of  Navy  Contracting,  Organization  Level 

(Apte  et  al.,  2008) 
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Equipment 

Acquisition  Planning 

4 

3 

3 

10 

Solicitation 

4 

3 

3 

10 

Source  Selection 

4 

3 

3 

10 

Contract  Administration 

2 

6 

2 

10 

Data  Processing  and 
Telecommunication 

Acquisition  Planning 

3 

1 

5 

0 

Solicitation 

3 

1 

5 

e 

Source  Selection 

3 

1 

5 

e 

Contract  Administration 

2 

2 

5 

» 

Utilities  and  Housekeeping 

Acquisition  Planning 

2 

2 

4 

8 

Solicitation 

2 

2 

4 

8 

Source  Selection 

2 

2 

4 

8 

Contract  Administration 

2 

2 

4 

8 

The  data  shows  that  the  Navy  used  competition  a  majority  of  the  time  in  all 
the  four  categories  of  services  (Apte  et  al.,  2008).  The  data  also  revealed  that  the  Air 
Force  contract  characteristics  were  very  similar  to  the  Navy’s.  The  study  showed 
that  a  majority  of  the  time,  firm  fixed-price  contracts  were  used;  however,  incentives 
were  not  used  very  often  (Apte  et  al.,  2008). 

The  Navy  used  a  project  team  approach  only  half  the  time  when  acquiring 
services  (Apte  et  al.,  2008).  Researchers  found  that  when  project  teams  were  used, 
they  were  led  only  by  project  managers  half  of  the  time.  The  research  also  found 
that  when  a  project  team  was  not  used,  either  the  customer  or  contract  officer 
managed  the  acquisition  effort  (Apte  et  al.,  2008). 

This  research  showed  common  characteristics  between  the  handling  of 
service  contracts  in  the  Navy  and  the  Air  Force,  such  as  the  use  of  firm  fixed-price 
contracts,  the  use  of  competition,  and  the  minimum  use  of  incentives  (Apte  et  al., 
2008).  The  research  revealed  no  standard  for  managing  service  contracts  in  the 
DoD.  It  showed  how  the  Air  Force  managed  its  contracts  at  the  installation  level, 
while  the  Navy  conducted  most  of  its  management  from  regional  offices.  The  study 
highlighted  the  different  ways  that  agencies  treat  service  contracts,  such  as  using 
project  teams.  The  research  showed  that,  even  when  project  teams  were  used,  the 
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contracting  officer  led  the  teams.  This  trend  has  the  potential  to  cause  problems 
because  the  lines  of  authority  become  grayer,  and  because  team  personnel  are  not 
working  directly  for  the  contracting  officer  (Apte  et  al. ,  2008). 

In  2009,  Apte,  Apte,  and  Rendon  published  their  fourth  study  of  service 
contracting.  This  research  widened  the  scope  of  the  survey  used  in  2008,  which 
focused  on  the  Air  Force  and  Navy,  to  include  the  Army.  This  allowed  the 
researchers  to  gain  a  better  understanding  of  how  the  agencies  in  the  DoD  manage 
their  services  differently.  The  2009  research  made  use  of  the  same  survey  that  was 
used  in  Apte  et  al.’s  (2008)  third  research  project  with  the  Navy  and  Air  Force.  This 
survey  was  distributed  to  eight  contracting  offices  which  included  40  Army 
installations.  There  was  a  75%  response  rate  to  this  survey.  The  services  that  the 
research  included  in  this  study  were  professional  and  administrative,  maintenance 
and  repair  of  equipment,  data  processing  and  telecommunication,  medical, 
maintenance  and  repair  of  equipment,  utilities  and  housekeeping,  and  transportation 
(Apte  et  al.,  2009). 

Apte  et  al.  (2009)  showed  that  the  Army  used  competitively  bid,  firm  fixed- 
price  contracts  without  incentives  in  a  majority  of  the  cases  (see  Table  3).  The  data 
also  showed  that  the  Army  conducted  a  majority  of  its  service  acquisition  at  the 
installation  level.  This  practice  may  have  an  impact  on  the  effectiveness  of  the 
contract  given  the  improved  ability  to  communicate  between  the  interested  parties 
(Apte  et  al.,  2009). 


Table  3.  Army  Contracting  Results 

(Apte  et  al.,  2009) 


Degree  of  Competition 

Contract  Type 

Contract  Incentive 

Competitive 

Sole 

Source 

N/A 

Fixed 

Cost 

N/A 

Incentive 

Fee 

Award 

Fee 

Award 

Term 

N/A 

FY03 

FY04 

FY05 

FY06 

FY07 

FY08 

58% 

10% 

32% 

58% 

8% 

34% 

0% 

6% 

2% 

02% 

50% 

10% 

31% 

60% 

7% 

33% 

0% 

5% 

1% 

03% 

50% 

10% 

30% 

60% 

7% 

33% 

1% 

5% 

1% 

03% 

62% 

10% 

28% 

63% 

8% 

20% 

1% 

5% 

2% 

02% 

65% 

10% 

25% 

65% 

8% 

27% 

1% 

6% 

2% 

01% 

\ 

10% 

24% 

66% 

8% 

26% 

1% 

6% 

2% 

01% 

The  study  revealed  that  the  Army  used  the  project  team  approach  62%  of  the 
time  (Apte  et  al.,  2009).  The  respondents  said,  however,  that  even  when  a  project 
team  approach  was  used,  the  contracting  officer  led  the  team  61%  of  the  time  (see 
Table  4).  As  mentioned  in  the  2008  Apte  et  al.  research,  this  situation  could  cause 
communication  problems  and  the  graying  of  authoritative  lines  because  the 
contracting  officer  does  not  have  real  authority  over  most  project  team  members. 
Another  finding  of  Apte  et  al.  (2009)  was  related  to  the  use  of  the  life-cycle 
approach;  this  approach  was  used  40%  of  the  time  for  routine  services.  For  non- 
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routine  services,  the  life-cycle  approach  was  only  used  21%  of  the  time.  The  use  of 
the  life-cycle  approach  should  be  used  to  ensure  proper  management.  If  services 
are  non-routine  in  nature,  this  approach  will  allow  for  higher  levels  of  uncertainty. 
This  uncertainty  and  increased  risk  will  be  compounded  when  it  is  not  mitigated 
through  the  use  of  the  life-cycle  approach  (Apte  et  al. ,  2009). 

This  research  also  found  that  most  acquisition  professionals  disagreed  that 
their  organizations  had  sufficient  positions  and  that  the  available  positions  were 
adequately  filled  (Apte  et  al.,  2009).  The  data  also  showed  that  only  39%  of  the 
respondents  believed  the  acquisition  workforce  was  trained,  and  only  45%  believed 
the  workforce  was  adequately  trained  (Apte  et  al.,  2009). 

Table  4.  Use  of  Project  Team 

(Apte  et  al.,  2009) 


No.  of 
Organiza¬ 
tions 

Organizations  Using  Project  Team  Approach 

SubT 

otal 

Who  leads  acquisition? 

Who  owns  requirements? 

Contracting 

Officer 

Other  (PM, 
QAE) 

Contracting 

Officer 

Customer 
(PM,  QAE) 

61 

38 

26 

12 

10 

28 

No.  of 
Organiza¬ 
tions 

Organizations  Not  Using  Project  Team  Approach 

SubT 

otal 

Who  leads  acquisition? 

Who  owns  requirements? 

Contracting 

Officer 

Other  (PM, 
QAE) 

Contracting 

Officer 

Customer 

(PM,  QAE) 

61 

23 

11 

12 

5 

18 

In  2010,  Apte,  Apte,  and  Rendon  completed  their  fifth  research  project.  This 
project  analyzed  the  data  that  was  collected  in  their  previous  studies  involving  the 
Air  Force,  Navy,  and  Army  to  gain  a  better  understanding  of  how  service  acquisition 
was  being  managed  across  the  DoD  (Apte  et  al.,  2010). 

This  research  first  examined  contract  characteristics  for  the  three  agencies. 
The  year  2007  was  the  focus  for  the  research  because  that  year  was  a  common 
time  period  of  data  collected  for  all  three  agencies  (Apte  et  al.,  2010).  As  seen  in 
Figure  13  and  discussed  in  earlier  reports,  full  and  open  competition  was  used  a 
majority  of  the  time. 
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Figure  13.  Contract  Characteristics  Between  Army,  Navy,  and  Air  Force 

(Apte  et  al.,  2010) 

Apte  et  al.  (2010)  reviewed  the  previous  data  and  compared  the  agencies. 

The  researchers  also  gave  recommendations  to  improve  the  systems  in  place.  One 
such  recommendation  was  for  the  Navy  to  adopt  a  more  disciplined  approach  and  to 
look  at  using  a  possible  virtual  project  team  because  the  acquisitions  and 
management  mainly  occur  at  the  regional  level  (Apte  et  al.,  2010).  If  this  were 
implemented,  the  quality  assurance  evaluator  (QAE)  or  contracting  officer 
representative  (COR)  can  serve  as  the  site  manager,  and  issues  and  communication 
can  improve  between  the  sites  of  the  services  and  where  the  contract  is  officially 
being  managed  (Apte  et  al.,  2010).  A  recommendation  for  the  Army  and  Air  Force 
was  to  look  at  establishing  an  installation  project  manager  who  is  overall  in  charge  of 
cost,  schedule,  and  performance  for  services  on  the  installation.  Apte  et  al.  (2010) 


\  rf*  *•'!  N7UM  / 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 


-30- 


recommended  for  all  of  the  agencies  to  increase  the  effectiveness  of  all  acquisition 
training  to  ensure  a  qualified  workforce. 

Apte,  Apte,  and  Rendon  completed  their  sixth  study  in  2012  on  service 
acquisitions.  This  research  focused  on  the  drivers  of  management  practices  in  the 
Army.  From  the  previous  research  they  conducted,  they  believed  that  four  factors 
had  major  impacts  on  service  contracting  (Apte  et  al. ,  2012).  These  factors  are  the 

•  type  and  quantity  of  services  being  outsourced  and  the  associated 
acquisition  related  workload; 

•  characteristics  of  contracts  being  awarded; 

•  capacity  available  to  carry  out  the  contracting,  project  management, 
and  surveillance  work;  and 

•  various  management  practices,  such  as  use  of  project  team  or  life 
cycle  approaches.  (Apte  et  al.,  2012,  p.  6) 

The  relationship  is  shown  in  Figure  14. 


Figure  14.  Contract  Relationships 

(Apte  et  al.,  2012) 

Contract  file  reviews  and  interviews  at  Mission  and  Installation  Contracting 
Commands  (MICCs)  were  conducted  to  complete  this  research  (Apte  et  al.,  2012). 
This  research  focused  on  four  product  service  codes  (PSCs): 
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•  R — Professional,  Administrative,  and  Management  Support  Services; 

•  J — Maintenance,  Repair,  and  Rebuilding  of  Equipment  Services; 

•  S — Utilities  and  Housekeeping  Services;  and 

•  D — Automatic  Data  Processing  and  Telecommunications  Services 
(Apte  et  al .,  2012,  p.  10). 

These  PSCs  account  for  60%  of  the  Army’s  spending  on  services  (Apte  et  al., 
2012).  The  research  focused  on  contract  characteristics  and  management  practices 
(Apte  et  al.,  2012).  These  two  areas  include  the  following: 

•  Contract  characteristics:  level  of  competition,  contract  type, 
award/incentive  fee,  contract  cost,  number  of  modifications,  award 
basis. 

•  Management  Practices:  use  of  Independent  Government  Estimate 
(IGE),  number  of  personnel  assigned,  use  of  a  team  approach, 
acquisition  leadership,  contract  award  time,  acquisition  plan, 
performance  work  statement  or  statement  of  work  (PWS/SOW),  price 
analysis,  price  negotiation  memorandum,  quality  assurance  plan, 
closeout  letter.  (Apte  et  al.,  2012,  p.  10) 

Their  research  showed  some  relationships  between  service  type  and  three 
contracting  characteristics  and  a  statistical  relationship  between  service  type  and 
five  management  practices  (Apte  et  al.,  2012).  These  relationships  are  shown  in 
Figure  15. 


Figure  15.  Relationship  Between  Service  Type,  Contract  Characteristics,  and 

Management  Practices 

(Apte  et  al.,  2012) 
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Apte  et  al.  (2012)  found  during  this  project  that  the  average  cost  of  PSCs  was 
higher  than  for  the  other  three  service  codes.  They  noted  that  service  codes  R  and  S 
had  a  higher  number  of  modifications  than  D  and  J.  They  found  that  R  and  S  used 
best  value  trade-off  contracts  a  majority  of  the  time,  while  D  and  J  used  lowest  price 
technically  acceptable  (LPTA)  50%  of  the  time.  They  observed  that  firm  fixed-price 
contracts  were  always  used,  but  only  once  with  incentives.  They  also  did  not  find 
any  relationship  with  the  solicitation  process  and  service  type  (Apte  et  al.,  2012). 

After  evaluating  the  14  management  practices,  the  researchers  found  five 
management  practices  which  had  a  statistical  relationship  with  the  service  type 
(Apte  et  al.,  2012).  They  also  found  that  if  the  contract  was  over  the  simplified 
acquisition  threshold,  the  contracts  had  IGEs  only  32%  of  the  time.  Apte  et  al.  (2012) 
found  that  the  number  of  people  assigned  to  PSCs  R  and  S  were  higher  than  for  the 
other  service  codes.  Apte  et  al.  (2012)  documented  that  the  contracting  officer  led 
the  acquisitions  of  the  services  in  146  out  of  the  154  contracts  that  they  researched, 
and  the  contracting  specialist  led  acquisitions  for  the  other  eight  contracts.  The 
researchers  also  stated  that  only  43%  of  the  contracts  they  reviewed  had  a  quality 
assurance  surveillance  place  (QASP;  Apte  et  al.,  2012). 

While  investigating  whether  the  MICC’s  capacity  to  conduct  acquisition  work 
affects  its  management  practices,  Apte  et  al.  (2012)  found  that  there  is  a  statistical 
relationship  between  capacity  and  management  practices.  They  found  that  the 
offices  did  not  have  the  required  number  of  authorized  people  to  conduct  acquisition 
functions.  They  also  found  that  MICC  personnel  lacked  the  proper  training 
certifications  (see  Table  5). 

Table  5.  Contracting  Workforce  Results 

(Apte  et  al.,  2012) 


Capacity 

Capacity 

MICC 

MICC 

MICC 

Mice 

MICC 

Mice 

Mice 

Category 

Subcategories 

Office  A 

Office  C 

Office  D 

Office  E 

Office  F 

Office  G 

Office  H 

BJets 

WanaEted 

100% 

88% 

CO 

58% 

100*/. 

100*/. 

100% 

Nod- uan  ailed 

0% 

84% 

106% 

47% 

117% 

86% 

S6% 

DAWIAI 

23% 

13% 

23% 

8% 

0% 

2% 

0% 

Certifraton 

dawia  n 

162% 

24% 

16% 

54% 

66% 

66% 

68% 

DAWIAHI 

lOOf. 

27% 

33% 

118% 

0% 

32% 

32% 

<  1  year 

18% 

14% 

7% 

0% 

14% 

10% 

4% 

bqpeneire 

1  -2  years 

18% 

43% 

12% 

1% 

23% 

3% 

2% 

2-3  yean 

10% 

16% 

7% 

9% 

34% 

19% 

21% 

>  3  vears 

55% 

17% 

74% 

90% 

29*/. 

68% 

73% 

The  researchers  found  that  the  MICCs  have  a  substantial  workload  with 
minimal  personnel,  training,  and  experience  (Apte  et  al.,  2012). 
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In  2012,  Hagan,  Spede,  and  Sutton  conducted  research  on  how  the  Navy 
stakeholders  determined  and  defined  success  in  service  contracting.  After  surveying 
168  key  stakeholders,  Hagen  et  al.  (2012)  found  that  all  the  stakeholders  tended  to 
use  outcome  related  factors  as  opposed  to  process  related  factors  when  asked  to 
define  and  measure  the  success  of  service  contracts.  They  recommended  that  Navy 
service  contract  success  should  be  defined  in  terms  of  both  the  outcomes  and 
processes.  Outcome  success  was  defined  and  measured  in  terms  of  the  cost, 
schedule,  and  performance  of  successful  service  contracts.  Cost  outcomes 
referenced  cost  overruns,  total  profits,  cost  reasonableness,  and  cost  fairness. 
Schedule  related  to  the  ability  to  meet  assigned  timelines  and  complete  the  contract. 
Performance  dealt  with  stakeholder  satisfaction  and  the  ability  to  perform  the 
statement  of  work  (Hagan  et  al.,  2012). 

Process  success  was  defined  in  terms  of  how  the  contract  was  administered 
and  planned  and  the  number  of  protests  it  received  for  the  definition  of  success 
(Hagan  et  al.,  2012).  The  measurement  of  process  success  was  determined  by  the 
communication  between  stakeholders  and  the  number  of  modifications  the  contract 
received.  Administration  dealt  with  the  total  workload  on  the  acquisition  team  and  the 
number  of  contracts  the  contracting  officer  currently  managed.  Planning  was  related 
to  the  ability  of  a  contracting  officer  to  turn  requirements  into  contractual  obligations. 
The  number  of  protests  and  number  of  modifications  were  the  quantitative  numbers 
associated  with  each.  Communication  assessed  the  amount  of  information  flow  and 
the  quality  of  communication  between  the  stakeholders.  A  diagram  of  Hagan  et  al.’s 
(2012)  research  questions  is  shown  in  Figure  16. 
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Figure  16.  Diagram  of  Survey  Questions 

(Hagan  et  al.,  2012) 

Therefore,  based  on  the  studies  already  mentioned,  this  research  uses  the 
CPARS  data  results  as  a  surrogate  definition  of  a  successful  service  contract, 
considering  no  definitive  definition  has  been  established  by  the  DoD,  and  the 
CPARS  captures  the  outcomes  for  both  defining  and  measuring  a  successful  service 
contract  by  Hagan  et  al.  (2012). 

G.  SUMMARY 

This  chapter  contained  a  review  of  the  available  literature  on  DoD  service 
contracting  processes.  This  chapter  presented  an  overview  of  the  service  contract 
management  process  by  providing  an  in-depth  description  of  the  seven  steps  and 
associated  sub-elements.  The  review  of  the  CPARS  database  in  this  chapter  looked 
at  the  information  that  CPARS  provides  to  the  user  and  who  is  responsible  for 
capturing  CPARS  data  throughout  the  service  contract  management  process.  Next, 
the  chapter  described  how  stakeholder  theory  explains  how  the  differing  goals  of  the 
different  stakeholders  can  lead  to  a  contract  that  provides  the  best  value  to  all 
parties.  The  current  issues  facing  CPARS  were  also  described  to  highlight  some  of 
the  current  problems  the  system  is  facing.  Previous  studies  relating  to  service 
acquisitions  were  also  explained  to  provide  context  for  why  CPAR  assessments  can 
be  used  as  a  surrogate  for  contract  success.  The  next  chapter  examines  the 
research  questions  for  this  project  and  provides  a  description  of  how  the  research 
will  be  conducted,  the  data  used  in  the  research,  and  the  type  of  analysis  used  in  the 
project. 
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III.  RESEARCH  METHODOLOGY 


A.  INTRODUCTION 

The  purpose  of  this  chapter  is  to  discuss  how  this  research  is  conducted  and 
how  a  successful  service  contract  is  defined  using  the  CPARS  data.  This  chapter 
begins  by  discussing  why  the  CPARS  database  is  used  in  deriving  a  definition  of  a 
successful  service  contract.  It  then  expounds  upon  what  data  is  retrieved  from  the 
CPARS  database  and  how  that  data  is  analyzed.  The  results  of  the  research  are 
revealed  in  Chapter  IV. 

B.  DEFINING  A  SUCCESSFUL  SERVICE  CONTRACT 

CPARS  data  will  be  used  as  a  surrogate  definition  of  a  successful  service 
contract  because  it  captures  the  outcomes  defined  by  Hagan,  Spede,  and  Sutton 
(2012)  for  both  measuring  and  defining  a  service  contract.  Quality,  schedule,  and 
cost  control  assessments  are  captured  in  the  CPARS  database  and  report  key 
aspects  of  a  contractor’s  performance  on  the  contract.  The  quality  rating  contains 
the  AO’s  assessment  of  the  contractor’s  qualitative  performance  and  compares  it  to 
the  requirements  stated  in  the  contract.  The  schedule  rating  contains  an  assessment 
of  the  contractor’s  ability  to  meet  schedules  outlined  in  the  contract  such  as  “task 
orders,  milestones,  delivery  schedules,  and  administrative  requirements”  (DoD, 

2011,  p.  A3-7).  The  cost  control  rating  contains  an  assessment  of  the  contractor’s 
ability  to  “forecast,  manage,  and  control  the  costs”  associated  with  conducting  their 
services  (DoD,  2011,  p.  A3-7).  The  business  relations  rating  contains  an 
assessment  of  the  contractor’s  ability  to  coordinate  their  business  activities,  such  as 
corporate  behavior,  customer  satisfaction,  management,  and  attitude  towards 
customers.  The  management  of  key  personnel  rating  contains  an  assessment  of  the 
contactors  ability  to  maintain  qualified  individuals  in  key  positions  as  outlined  in  the 
contract.  The  utilization  of  small  business  rating  contains  an  assessment  of  the 
contractors’  ability  to  integrate  small  businesses  in  the  execution  of  the  contract. 
These  ratings  are  completed  and  approved  in  the  CPARS  database.  Once  the 
CPARS  ratings  are  approved,  the  information  is  stored  and  accessed  through  the 
PPIRS-RC  database.  This  research  obtained  the  CPARS  data  by  accessing  the 
PPIRS-RC  database  (DoD,  2011). 

C.  DESIGNING  THE  DATABASE 

The  search  of  the  PPIRS-RC  website  was  limited  to  the  Army  MICC  non¬ 
system  business  sector.  From  the  results  of  the  search,  a  spreadsheet  was  created 
of  all  the  contracts  contained  within  the  PPIRS-RC  database.  This  search  yielded 
14,395  contracts  in  total,  with  periods  of  performance  end  dates  ranging  from  1996 
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through  2013.  The  data  was  then  narrowed  to  only  those  contracts  containing  a 
Product  or  Service  Code  or  Federal  Service  Code  (PSC/FSC)  of  R,  D,  J,  or  S,  which 
yielded  5,621  contracts.  It  is  important  to  note  that  construction  contracts  were  not 
included  in  this  database  since  they  are  categorized  as  a  distinct  business  sector 
within  the  PPIRS-RC  database.  The  data  was  further  narrowed  by  only  searching 
five  contracting  organizations.  These  organizations  were  Fort  Eustis  (W91  ISO),  Fort 
Knox  (W9124D),  Fort  Hood  (W91 151 ),  Fort  Bragg  (W91247),  and  Fort  Sam  Houston 
(W91 1SE).  These  organizations  were  selected  because  they  are  organized  as  the 
MICC  Field  Directorate  Offices  (FDOs)  of  five  different  areas  of  focus,  as  seen  in 
Figure  17.  This  narrowed  the  data  to  715  distinct  service  contracts  that  were  used  in 
conducting  analysis,  as  seen  in  Table  6.  For  each  contract,  data  was  collected  on 
the  awarded  dollar  value,  basis  of  award,  type  of  contract,  quality  of  product/service 
rating,  schedule  rating,  cost  control  rating,  business  relations  rating,  management  of 
key  personnel  rating,  and  utilization  of  small  business  rating  in  order  to  answer  the 
four  primary  research  questions. 
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Table  6.  Database  Breakdown 


Total  Contracts 


Total  Army  MICC  Non- 
System  Contracts 

14395 

Removed  Non  R,  J,  S,  D 

Service  Contracts 

8774 

Total  R,  J,  S,  D  Service 

Contracts 

5621 

Removed  other  MICC  R,  J, 

S,  D  Service  Contracts 

4906 

MICC  FDO  Eustis,  Knox, 

Hood,  Bragg,  Sam 

Houston  Service 

Contracts 

715 

Fort  Eustis 

238 

Fort  Knox 

119 

Fort  Hood 

114 

Fort  Bragg 

55 

Fort  Sam  Houston 

189 

Once  the  database  was  established,  it  was  further  refined  by  labeling  each 
contract  as  a  success  or  a  failure.  Labeling  a  contract  as  a  failure  was  determined  by 
whether  a  contract  received  a  marginal  or  unsatisfactory  rating  in  quality  of 
product/service,  schedule,  cost  control,  business  relations,  management  of  key 
personnel,  or  utilization  of  small  business  rating.  Receiving  a  marginal  or 
unsatisfactory  rating  in  any  one  of  these  ratings  labels  the  entire  contract  as  a 
failure.  Those  contracts  not  determined  to  be  failures  were  labeled  as  successful 
service  contracts.  See  appendix  A  for  more  details. 

The  data  obtained  from  PPIRS-RC  was  exclusively  related  to  outcome 
successes  defined  by  Hagan  et  al.  (2012).  In  addition  to  the  PPIRS-RC  outcome- 
related  data,  organizational  data  was  collected  in  the  areas  of  annual  workload  in 
dollars,  number  of  actions  completed  by  the  MICC,  number  of  1 102  billets 
authorized,  and  percent  of  1 102  billets  filled.  This  data  was  added  to  the  database  in 
an  effort  to  analyze  some  of  the  process  success  characteristics  defined  by  Hagan, 
et  al.  (2012).  Once  the  database  was  completed,  it  was  analyzed  to  answer  the  four 
research  questions.  The  next  section  describes  how  the  database  is  analyzed  to 
answer  the  research  questions. 

D.  ANALYZING  THE  DATABASE 

Determination  of  whether  the  types  of  services  being  acquired  affected  the 
success  of  the  contract  was  conducted  by  determining  the  total  number  of  contracts 
in  each  of  the  R,  J,  S,  and  D  type  services  and  the  total  that  were  determined  to  be 
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failures.  The  total  failures  for  each  type  of  service  were  divided  by  the  total  number 
of  the  contracts  for  that  type  of  service.  This  result  was  the  failure  rate  for  that 
specific  type  of  service.  This  analysis  revealed  whether  one  type  of  service  resulted 
in  a  higher  failure  rating  than  other  types  of  services.  These  results  were  then 
analyzed  to  determine  the  stated  reasons  that  the  contracts  were  given  a  marginal 
or  unsatisfactory  rating  in  order  to  determine  whether  a  failure  pattern  emerged  for 
the  types  of  services  being  acquired.  Finally,  additional  analysis  was  conducted  to 
attempt  to  identify  other  data  correlations  that  may  highlight  why  a  certain  service 
type  was  evaluated  as  a  failure.  These  results  are  useful  in  determining  the 
management  levers  that  might  assist  in  ensuring  a  successful  service  contract. 
Determination  of  whether  contractual  amounts  affected  the  success  of  a  service 
contract  was  conducted  in  the  same  manner  as  the  types  of  services  being  acquired. 
The  contract  categories  in  this  analysis  were  broken  down  into  five  separate 
categories:  0— $1  million,  greater  than  $1  million— $1 0  million,  greater  than  $10 
million-$50  million,  greater  than  $50  million— $1  billion,  and  greater  than  $1  billion. 
The  failure  rates  were  then  determined  for  each  of  these  categories  and  whether  a 
correlation  existed  between  the  ratings  that  the  contracts  received.  Each  category 
then  received  an  in-depth  analysis  of  why  it  received  a  failure  rating  by  showing 
which  category  (quality,  schedule,  cost  control,  business  relations,  management  of 
key  personnel,  and  utilization  of  small  business)  received  a  marginal  or 
unsatisfactory  rating. 

To  answer  the  third  question  about  whether  the  level  of  competition  used 
affects  the  success  of  a  service  contract,  the  contracts  were  separated  into  three 
different  categories:  competitive,  non-competitive,  and  other.  Failure  rates  were 
used  for  each  of  these  categories  to  reveal  whether  the  level  of  competition  used 
was  related  to  the  failure  of  that  level  of  competition.  The  ratings  of  the  unsuccessful 
contracts  were  then  analyzed  to  determine  which  area  (quality,  schedule,  cost 
control,  business  relations,  management  of  key  personnel,  and  utilization  of  small 
business)  was  more  frequently  associated  with  the  contract  being  rated  as  marginal 
or  unsatisfactory. 

The  final  research  question  focused  on  the  type  of  contract  used  and  the 
success  of  the  contract.  This  required  dividing  the  contracts  into  six  different 
categories.  The  first  four  categories  are  cost  plus  award  fee  (CPAF),  cost  plus  fixed 
fee  (CPFF),  cost  plus  incentive  fee  (CPIF),  and  firm  fixed  price  (FFP).  The  remaining 
two  categories  are  a  combination  and  other.  The  combination  category  contains 
contracts  that  used  a  combination  of  CPAF,  CPFF  and  FFP  together  instead  of 
using  just  one  type.  The  other  category  includes  all  the  contracts  that  did  not  fit  into 
the  previous  five  categories,  such  as  labor  hours  or  time  and  materials.  The  failure 
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rates  and  associated  reasons  determined  whether  the  types  of  contracts  being  used 
affected  the  success  of  a  service  contract. 

Once  the  primary  research  questions  were  analyzed,  analysis  on  the  location, 
duration,  year  the  contract  was  completed,  and  the  MICC  organizational  data  was 
used  to  determine  whether  any  of  these  additional  factors  relate  to  the  four  contract 
variables.  The  MICC  organizational  data  focuses  on  four  main  areas:  annual 
workload  in  dollars  spent,  number  of  completed  contracts,  number  of  authorized 
1102  billets,  and  the  percent  of  1 102  billets  filled.  This  organizational  data  proved 
useful  in  adding  some  of  the  process  success  variables  defined  by  Hagan  et  al. 
(2012)  to  the  analysis.  The  failure  rates  and  rating  patterns  among  these  contracts 
provided  additional  insight  into  the  four  primary  research  questions. 

Answering  these  research  questions  with  the  data  collected  reveals  whether 
the  four  contract  variables  had  a  statistically  significant  impact  on  the  success  or 
failure  of  a  service  contract.  The  results  of  that  analysis  help  focus  further  research 
to  explain  why  that  is  the  case  and  what  the  DoD  can  do  to  increase  the  success 
rate  of  service  contracts. 

Figure  18  illustrates  the  research  methodology  previously  described.  The 
column  on  the  right  contains  the  six  CPARS  data  areas.  These  areas  are  used  to 
label  each  contract  as  a  success  or  a  failure  based  on  the  ratings  (marginal  or 
unsatisfactory).  The  contracts  labeled  as  failures  are  analyzed  using  the  four 
contract  variables  shown  on  the  left  column.  The  purpose  is  to  determine  whether 
there  is  a  relationship  between  contract  variables  and  contract  success. 


Contraa  Variables 


CPARS  Data 


Figure  18.  Research  Methodology 


E.  SUMMARY 

This  chapter  reviewed  the  research  methodology  used  in  answering  the  four 
research  questions.  It  began  by  defining  a  successful  service  contract  according  to 
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this  research.  The  chapter  then  described  how  the  database  was  designed  to 
capture  the  data  that  was  analyzed.  Finally,  the  chapter  discussed  how  the  research 
questions  were  answered.  The  next  chapter  shows  the  data  analysis  and  the  results 
of  the  research. 
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IV.  ANALYSIS  AND  IMPLICATIONS 


A.  INTRODUCTION 

This  chapter  reveals  the  results  of  the  research  into  what  variables  drive 
successful  service  contracts?  This  chapter  begins  with  an  overview  of  the  database 
created  for  this  report  in  order  to  create  a  clear  picture  of  how  the  results  were 
derived.  This  overview  includes  the  total  number  of  successful  and  unsuccessful 
contracts  separated  into  specific  categories.  Once  the  database  is  framed,  the 
chapter  explores  the  four  primary  research  questions  driving  this  study,  which  focus 
on  the  failure  rates  of  the  different  categories.  The  chapter  then  explores  additional 
information  obtained  from  the  database,  including  the  MICC  organizational  data,  in 
order  to  add  additional  insight  into  the  four  primary  variables.  The  additional  analysis 
enhances  the  understanding  of  the  drivers  of  success  and  adds  valuable  insight  into 
what  makes  service  contracts  successful.  This  chapter  concludes  with  a  significance 
test  and  the  implications  of  the  findings. 

B.  OVERVIEW  OF  THE  DATA 

The  database  consisted  of  715  contracts  pulled  from  the  PPIRS-RC 
database.  These  contracts  were  from  five  different  MICC  regional  offices  and 
consisted  of  R  (Professional,  Administrative,  and  Management  Support  Services),  J 
(Maintenance,  Repair,  and  Rebuilding  of  Equipment  Services),  S  (Utilities  and 
Housekeeping  Services),  and  D  (Automatic  Data  Processing  and 
Telecommunications  Services)  type  of  services.  The  data  pulled  represent 
population  data  that  meet  the  criteria  of  the  MICC  regional  offices  and  the  services 
previously  listed.  Labeling  a  contract  as  a  failure  was  determined  by  whether  a 
contract  received  a  marginal  or  unsatisfactory  rating  in  quality  of  product/service, 
schedule,  cost  control,  business  relations,  management  of  key  personnel,  or 
utilization  of  small  business  rating.  Receiving  a  marginal  or  unsatisfactory  rating  in 
any  one  of  these  ratings  labels  the  entire  contract  as  a  failure.  Those  contracts  not 
determined  to  be  failures  were  labeled  as  successful  service  contracts.  The  data 
consisted  of  715  contracts;  22  resulted  in  a  failure  label  as  just  described  and  as 
explained  in  Chapter  III.  This  resulted  in  a  total  contract  failure  rate  of  3.08%.  These 
results  are  clearly  seen  in  Figure  19  and  Table  7. 
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Total  Successes  and  Failures 

22 

3.08%  \ 

y Successes 

y  Failures 

V 

\ 

693 

96.92% 

Figure  19.  Total  Contracts 


Table  7.  Total  Contract  Information 


Failures 

Success 

Total 

Failure 

Rate 

Contracts 

22 

693 

715 

3.08% 

Contracts  evaluated  and  entered  in  CPARS  are  evaluated  by  quality, 
schedule,  cost  control,  business  relations,  management  of  key  personnel,  and 
utilization  of  small  business  (discussed  in  more  detail  in  Chapter  III).  The  total 
contract  failures  for  each  area  of  evaluation  can  be  seen  in  Figure  20.  Each  area  of 
evaluation  was  given  a  score  respective  of  the  rating  the  evaluator  gave  that  area 
(see  Table  8).  The  areas  of  evaluation  scores  were  then  averaged  to  examine  what 
areas  where  rated  higher  throughout  the  population  (see  Table  9).  Business 
relations  had  the  highest  average  among  the  other  evaluation  areas  with  an  average 
score  of  3.  Management  of  key  personnel  had  the  next  highest  average  with  a  score 
of  2.68.  Quality,  schedule,  and  utilization  of  small  business  had  equal  average 
scores  of  2.5.  Cost  control  had  the  lowest  average  score  of  all  the  areas  with  a  2.31 
average  score. 


Table  8.  Area  of  Contract  Evaluation  Scores 


Rating 

Score 

Exceptional 

5 

Very  Good 

4 

Satisfactory 

3 

Marginal 

2 

Unsatisfactory 

1 
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Table  9.  Average  Success  and  Failure  Rates  of  PPIRS  Areas  of  Contract 

Evaluation 


Successes 

Average 

Ratings 

Failures 

Average 

Ratings 

Quality 

4.19 

2.5 

Schedule 

4.19 

2.5 

Cost  Control 

4.1 

2.31 

Business  Relations 

4.17 

3 

Management  of  Key  Personnel 

4.18 

2.68 

Utilization  of  Small  Business 

4.07 

2.5 

Figure  20.  Stated  Reason  of  Failure  Label  for  All  Contracts 
C.  ANSWERING  THE  RESEARCH  QUESTIONS 
1.  Research  Questions  Restated 

The  primary  question  addressed  in  this  research  concerns  what  variables 
drive  successful  service  contracts.  This  research  focuses  on  four  research  questions 
that  explore  the  variables  that  result  in  successful  service  contracts.  These 
questions  are  as  follows: 

•  Do  the  types  of  services  being  acquired  affect  the  success  of  a  service 
contract? 

•  Do  the  contractual  amounts  affect  the  success  of  a  service  contract? 
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•  Does  the  level  of  competition  used  affect  the  success  of  a  service 
contract? 

•  Does  the  contract  type  affect  the  success  of  a  service  contract? 

This  information  proves  valuable  in  identifying  the  variables  that  ensure  a 
successful  service  contract. 

2.  Type  of  Service 

The  first  research  question  is  as  follows:  Do  the  types  of  services  being  acquired 
affect  the  success  of  a  service  contract?  To  answer  this  question,  four-product 
service  codes  were  examined  using  data  gathered  from  the  PPIRS  database. 
Contracts  with  a  product  service  code  of  R  (Professional,  Administrative,  and 
Management  Support  Services),  J  (Maintenance,  Repair,  and  Rebuilding  of 
Equipment  Services),  S  (Utilities  and  Housekeeping  Services),  and  D  (Automatic 
Data  Processing  and  Telecommunications  Services)  were  analyzed  to  include  the 
failure  rates  of  each  service  contract.  The  database  contained  331  R-type  contracts, 
58  J-type  contracts,  292  S-type  contracts,  and  34  D-type  contracts  (see  Table  8). 

Finding  1 :  The  S-type  contracts  had  the  highest  failure  rate  of  all  the 
product  service  codes  analyzed. 

The  1 1  labeled  failures  of  S-type  contracts  resulted  in  a  3.77%  failure  rate 
(see  Figure  21 ).  There  were  two  reasons  that  tied  for  the  most  common  reasons  for 
S-type  contract  failures.  These  reasons  were  six  business-relation  failures  and  six 
failures  due  to  management  of  key  personnel  (see  Figure  22).  R-type  contracts  had 
nine  labeled  failing  contracts  out  of  331 ,  giving  R-type  contracts  a  failure  rate  of 
2.72%  (see  Figure  21),  which  was  the  second  lowest.  The  most  common  reason 
entered  into  the  PPIRS  database  was  quality.  J-type  contracts  consisted  of  58 
contracts  with  two  labeled  failures.  This  gave  the  J-type  contracts  a  failure  rate  of 
3.45%.  For  both  labeled  failures  of  the  J-type  contract,  scheduling  was  listed  as  a 
reason  for  both  of  these  failures.  There  were  only  34  of  D  service  code  contract 
types  with  0  failures  (see  Figures  21  and  22). 


Table  10.  Type  of  Service  Acquired  Total  Successes  and  Failures 


Type  of  Service 
Acquired 
Categories 

Total 

Successes 

Total 

Failures 

Total 

Contracts 

Failure 

Rates 

D 

34 

0 

34 

0.00% 

J 

56 

2 

58 

3.45% 

R 

322 

9 

331 

2.72% 

S 

281 

11 

292 

3.77% 

Total 

693 

22 

715 

3.08% 
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Figure  21.  Failure  Rates  of  the  Different  Product  Service  Code  Contracts 


1  Quality 

'Schedule 

1  Cent  Control 

1 1  Bounce  c  Relatione 

1  Management  of  Key 
Personnel 
1  Utilization  of  Small 
Bounces 


Figure  22.  Reasons  for  Product  Service  Code  Failure 
3.  Contractual  Amounts 

The  second  research  question  is  as  follows:  Do  the  contractual  amounts 
affect  the  success  of  a  service  contract?  To  answer  this  question  the  research 
grouped  the  contracts  into  dollar  amounts.  The  different  dollar  amount  categories 
contracts  that  were  0— $1  million,  contracts  greater  than  $1  million— $1 0  million, 
contracts  greater  than  $10  million-$50  million,  contracts  greater  than  $50  million— $  1 
billion,  and  contracts  greater  than  $1  billion  (see  Table  1 1 ).  The  researchers  looked 
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at  these  different  dollar  amounts  separately  and  examined  the  failure  rate  of  each 
group. 


Finding  2:  The  contract  grouping  that  consisted  of  contracts  worth  a 
dollar  amount  greater  than  $50  million-$1  billion  had  the  highest  failure  rate  of 
all  the  groups  in  the  analysis. 

This  group  consisted  of  92  contracts  with  eight  labeled  failures,  giving  the 
group  a  failure  rate  of  8.7%  (see  Figure  23).  This  group’s  most  common  reason  for 
failing  was  cost  control  (see  Figure  24).  This  reason  was  listed  for  six  failed 
contracts.  This  failure  rate  is  much  higher  than  the  total  contract  average  failure  rate 
of  3.08%.  In  the  first  group  that  consisted  of  contracts  that  were  worth  0— $1  million, 
there  were  a  total  of  35  contracts.  In  this  first  group,  there  was  only  one  labeled  a 
failure  (see  Figure  23).  This  gave  this  group  a  2.86%  failure  rate.  This  contract  was 
labeled  a  failure  because  of  quality  (see  Figure  24).  The  group  consisting  of 
contracts  greater  than  $1-50  million  was  the  largest  of  all  the  grouped  dollar 
amounts.  It  consisted  of  466  contracts,  and  of  those,  10  were  labeled  failures.  That 
gave  this  group  a  2.15%  failure  rate  (see  Figure  23).  Although  this  group  had  the 
most  failures  numerically,  it  still  was  under  the  average  failure  rate  because  of  the 
amount  of  contracts  total  in  this  group.  The  most  common  reason  for  this  group  to 
fail,  according  to  PPIRS,  was  for  quality;  it  was  cited  seven  times.  The  contracts 
greater  than  $10  million-$50  million  group  consisted  of  1 18  contracts.  There  were 
three  labeled  failures  in  this  group.  This  group  had  a  2.54%  failure  rate.  This  group 
was  also  under  the  average  total  contract  failure  rate  of  3.08%.  This  contract  group 
most  commonly  failed  for  scheduling  issues  and  management  of  key  personnel.  It 
failed  twice  for  scheduling  twice  and  management  of  key  personnel  (see  Figure  24). 
This  means  that  one  of  the  contracts  in  this  group  had  both  issues  listed  as  reasons 
for  failure.  The  group  consisting  of  contracts  worth  greater  than  $1  billion  was  the 
smallest  group  in  the  contractual  amount  grouping.  It  consisted  of  only  four  contracts 
and  did  not  contain  any  labeled  failures. 


Table  1 1 .  Contract  Amount  Total  Successes  and  Failures 


Contract  Dollar 
Amount 
Categories 

Total 

Successes 

Total 

Failures 

Total 

Contracts 

Failure 

Rates 

$0-$1  M 

34 

1 

35 

2.86% 

>$1  M-$10M 

456 

10 

466 

2.15% 

>$10M-$50M 

115 

3 

118 

2.54% 

>$50M-$1  B 

84 

8 

92 

8.70% 

>$1  B 

4 

0 

4 

0.00% 

Total 

693 

22 

715 

3.08% 
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Figure  23.  Failure  Rate  by  Grouped  Dollar  Value 


Figure  24.  Reasons  for  Grouped  Dollar  Value  Failure 
4.  Level  of  Competition 

The  third  research  question  is  as  follows:  Does  the  level  of  competition  affect 
the  success  of  a  service  contract?  To  answer  this  question,  the  research  grouped 
the  contracts  into  three  categories:  competitive,  non-competitive,  and  other.  The 
researchers  looked  at  these  different  categories  separately  and  examined  the  failure 
rate  of  each  group.  These  categories  are  shown  in  Table  12. 
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Finding  3:  Contracts  competed  competitively  had  the  highest  failure  rate 
when  compared  to  the  other  two  forms  of  competition  available. 

There  were  540  competitive  contracts  examined  in  the  database.  Of  these 
540  contracts,  17  were  labeled  a  failure,  which  yields  a  failure  rate  of  3.15%.  The 
reasons  that  most  often  resulted  in  a  contract  failure  were  in  the  areas  of  schedule 
and  cost  control,  which  were  each  referenced  seven  times.  The  next  highest 
referenced  source  of  failure  was  management  of  key  personnel,  which  was 
referenced  six  times.  The  failure  rates  due  to  level  of  competition  are  shown  in 
Figure  25.  Non-competitive  contracts  had  the  next  highest  failure  rate  at  2.91%. 
There  were  172  non-competitive  contracts  in  the  database,  of  which  five  were 
labeled  failures.  Quality  was  referenced  four  times  while  schedule,  management  of 
key  personnel,  and  business  relations  were  each  referenced  twice.  Contracts 
competed  as  other  had  three  contracts  in  the  database  with  zero  labeled  failures. 
The  rating  statistics  across  the  levels  of  competition  are  shown  in  Figure  26. 


Table  12.  Level  of  Competition  Total  Successes  and  Failures 


Level  of  Competition 
Categories 

Total 

Successes 

Total 

Failures 

Total 

Contracts 

Failure 

Rates 

Basis  (Competitive) 
Basis  (Non- 

523 

17 

540 

3.15% 

Competitive) 

167 

5 

172 

2.91% 

Basis  (Other) 

3 

0 

3 

0.00% 

Total 

693 

22 

715 

3.08% 

N 

O 

£ 

o 

- 

3 


3.50% 

3.00% 

2.50% 

2.00% 

1.50% 

1.00% 

0.50% 

0.00% 


3.15% 


2.910/0 


Basis  (Competitive) 


Basis  (Non- 
Competitive) 

Basis  of  Contract  Award 


0.00% 

Basis  (Other) 


Figure  25.  Failure  Rates  Among  Level  of  Competition 
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Figure  26.  Levels  of  Competition  Stated  Reasons  for  Failure 

5.  Contract  Type 

The  fourth  research  question  is  as  follows:  Does  the  contract  type  affect  the 
success  of  a  service  contract?  To  answer  this  question,  the  researchers  grouped  the 
contracts  into  six  categories:  CPAF,  combination,  CPFF,  CPIF,  FFP,  and  other.  The 
researchers  looked  at  each  of  these  categories  separately  and  examined  the  failure 
rate  of  each  group,  as  seen  in  Table  13. 

Finding  4:  Contracts  structured  as  a  combination  contract  had  the 
highest  failure  rate  when  compared  to  the  other  five  types  of  available 
contracts. 

There  were  four  combination  contracts  examined  in  the  database.  Of  these 
four  contracts,  two  were  labeled  failures,  which  yields  a  failure  rate  of  50.0%. 
Schedule  and  cost  were  both  referenced  twice  in  the  failed  contracts  while  quality 
and  management  of  key  personnel  were  each  referenced  once.  The  failure  rates 
due  to  contract  type  are  shown  in  Figure  27.  CPFF  contracts  had  the  next  highest 
failure  rate  at  5.56%.  There  were  36  CPFF  contracts  in  the  database,  of  which  two 
were  labeled  failures.  Cost  control  was  referenced  twice  and  schedule  was 
referenced  once  in  the  failed  contracts.  Contracts  competed  as  CPAF  had  58 
contracts  in  the  database  with  three  of  them  labeled  as  failures.  This  yielded  a  failure 
rate  of  5.17%.  Two  of  these  failed  contracts  referenced  cost  control  and  business 
relations  while  one  referenced  the  management  of  key  personnel.  FFP  contracts  had 
524  contracts  in  the  database  with  14  of  them  labeled  as  failures.  This  yielded  a 
failure  rate  of  2.68%.  Seven  of  these  failed  contracts  referenced  quality  while  six 
referenced  the  management  of  key  personnel.  Other  contract  types  had  89  contracts 
in  the  database  with  one  labeled  as  a  failure  because  of  quality  and  schedule,  which 
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yielded  a  failure  rate  of  1.12%.  There  were  four  CPIF  contracts,  which  had  zero 
labeled  failures.  The  rating  statistics  among  contract  types  are  shown  in  Figure  28. 


Table  13.  Contract  Type  Total  Successes  and  Failures 


Contract  Type 
Categories 

Total 

Successes 

Total 

Failures 

Total 

Contracts 

Failure 

Rates 

Type  (CPAF) 

55 

3 

58 

5.17% 

Type  (Combination) 

2 

2 

4 

50.00% 

Type  (CPFF) 

34 

2 

36 

5.56% 

Type  (CPIF) 

4 

0 

4 

0.00% 

Type  (FFP) 

510 

14 

524 

2.67% 

Type  (Other) 

88 

1 

89 

1.12% 

Total 

693 

22 

715 

3.08% 

O 

— 

£ 


60 

50 

40 


£  30 


(2 


20 

10 

0 


00% 

00% 

00% 

00% 

00% 

00%  5.17% 

00%  1 - 1 


50.00% 


vy 


5.56% 


0.00% 


2.68% 


1.12% 


v$> 


Contract  Type 


Figure  27.  Contract  Type  Failure  Rate 
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Figure  28.  Contract  Type  Stated  Reasons  for  Failure 
D.  ADDITIONAL  ANALYSIS 

1.  Analysis  Overview 

The  next  section  shows  the  results  from  the  additional  analysis  the  PPIRS 
database  revealed.  It  includes  the  failure  rates  of  the  MICCs,  duration  of  contracts, 
and  the  MICC  organizational  data.  The  organizational  data  includes  the  annual 
workload,  number  of  actions,  number  of  assigned  1102  billets,  and  the  percent  of 
those  billets  filled. 

2.  MICCs 

The  contracts  examined  came  from  the  five  different  MICC  regions  described 
in  Chapter  III.  The  Fort  Bragg  regional  office  had  55  contracts,  Fort  Sam  Houston 
had  189  contracts,  Fort  Knox  had  119  contracts,  Fort  Hood  had  114  contracts,  and 
Fort  Eustis  had  238  contracts.  The  MICC  regions  and  their  associated  data  are 
shown  in  Table  12. 

Finding  5:  Contracts  completed  by  Fort  Hood  had  the  highest  failure 
rate  compared  to  the  other  four  MICC  regions. 

There  were  114  contracts  completed  by  Fort  Hood  examined  in  the  database. 
Of  these  114  contracts,  six  were  labeled  failures.  This  yields  a  failure  rate  of  5.26%. 
The  reason  that  most  often  resulted  in  a  contract  being  labeled  a  failure  was  in  the 
area  of  quality,  which  was  referenced  four  times.  Schedule  and  business  relations 
were  each  referenced  three  times.  The  failure  rates  across  the  MICCs  are  shown  in 
Figure  29.  Fort  Knox  had  the  next  highest  failure  rate  at  5.04%.  There  were  119 
contracts  from  Fort  Knox  in  the  database,  of  which  six  were  labeled  failures.  The 
reason  stated  for  these  contracts  was  most  frequently  in  the  area  of  management  of 
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key  personnel,  which  was  referenced  three  times.  Quality,  schedule,  cost  control, 
and  business  relations  were  each  referenced  twice.  Fort  Sam  Houston  had  189 
contracts  in  the  database  with  eight  of  them  being  labeled  failures.  This  yielded  a 
failure  rate  of  4.23%.  The  most  commonly  stated  reason  for  a  contract  being 
determined  a  failure  at  Fort  Sam  Houston  was  in  the  area  of  cost  control  with  five 
ratings.  Schedule  was  the  next  most  frequent  rating  that  resulted  in  a  failure,  which 
was  referenced  four  times.  Fort  Bragg  had  a  failure  rate  of  1 .82%  and  included  55 
contracts  (with  one  labeled  failure).  The  only  rating  that  received  a  failure  label  from 
Fort  Bragg  was  in  the  area  of  quality,  but  this  result  was  driven  from  only  one 
contract.  Fort  Eustis  had  the  most  contracts  in  the  database  at  238  (with  only  one 
labeled  failure),  but  also  had  the  lowest  failure  rate  (0.42%)  of  any  of  the  MICC 
contracts  examined.  The  rating  statistics  across  the  MICCs  are  shown  in  Figure  30. 


Table  14.  MICCs  Total  Successes  and  Failures 


MICC  Categories 

Total 

Successes 

Total 

Failures 

Total 

Contracts 

Failure 

Rates 

Fort  Bragg 

54 

1 

55 

1 .82% 

Fort  Sam  Houston 

181 

8 

189 

4.23% 

Fort  Knox 

113 

6 

119 

5.04% 

Fort  Hood 

108 

6 

114 

5.26% 

Fort  Eustis 

237 

1 

238 

0.42% 

Total 

693 

22 

715 

3.08% 

6.00% 


Houston 

MICCs 


Figure  29.  Failure  Rates  Among  the  MICCs 


VrfTi  vtivm  rn  *;iS7um7 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 


-54- 


Figure  30.  MICCs  Stated  Reason  for  Failures  Among  the  Ratings 

3.  Duration 

The  duration  of  contracts  span  from  zero  to  1,246  days  and  occurred  between 
2006  and  2013.  The  days  were  broken  into  five  different  categories,  each  containing 
90  days.  The  final  category  contains  contracts  lasting  over  365  days  in  duration.  In 
the  first  category,  zero  to  90  days,  there  were  27  contracts.  There  were  25  contracts 
lasting  between  91  and  180  days.  The  181  to  270  category  contained  52  contracts. 
The  271  to  365  day  category  contained  the  largest  amount  of  contracts,  at  542. 
There  were  69  contracts  lasting  over  365  days.  These  numbers  are  clearly  seen  in 
Table  13. 

Finding  6:  Contracts  with  a  duration  between  0  and  90  days  had  the 
highest  failure  rate  compared  to  the  other  four  contract  durations  analyzed. 

There  were  27  0-90-day  contracts  examined  in  the  database.  Of  these  27 
contracts,  two  were  labeled  failures  as  defined  within  the  parameters  of  this 
research,  which  yields  a  failure  rate  of  7.41%.  Quality,  schedule,  and  cost  control 
were  each  referenced  once  in  the  failed  contracts.  The  failure  rates  because  of 
contract  duration  are  shown  in  Figure  31 . 271 -365-day  contracts  had  the  next 
highest  failure  rate  at  3.32%.  There  were  542  271-365-day  contracts  in  the 
database,  of  which  18  were  labeled  failures.  Quality,  cost  control,  and  the 
management  of  key  personnel  were  each  referenced  seven  times  while  schedule 
and  business  relations  were  each  referenced  six  times  in  the  failed  contracts.  181  — 
270-day  contracts  had  52  contracts  in  the  database  with  one  labeled  failure.  This 
yielded  a  failure  rate  of  1 .92%.  The  failed  contract  referenced  schedule,  business 
relations,  and  the  management  of  key  personnel.  Contracts  in  excess  of  365  days 
had  69  contracts  in  the  database  with  one  labeled  failure.  This  yielded  a  failure  rate 
of  1 .45%.  The  failed  contract  referenced  quality  and  schedule.  Contracts  between 
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91-180  days  had  25  contracts  in  the  database  with  no  labeled  failures.  The  rating 
statistics  among  contract  types  are  shown  in  Figure  32. 


Table  15.  Duration  Total  Successes  and  Failures 


Contract 

Duration 

Categories 

Total 

Successes 

Total 

Failures 

Total 

Contracts 

Failure 

Rates 

0-90 

25 

2 

27 

7.41% 

91-180 

25 

0 

25 

0.00% 

181-270 

51 

1 

52 

1 .92% 

271-365 

524 

18 

542 

3.32% 

>365 

68 

1 

69 

1 .45% 

Total 

693 

22 

715 

3.08% 

8.00%  ■)  7.41% 

7.00% 

6.00% 

S  5.00% 

n 

%  4.00% 

5  3.00% 

2.00% 

1.00% 

0.00% 

0-90 


3.32% 


91-180  181-270  271-365  >365 

Contract  Ouration  in  Days 


Figure  31.  Contract  Duration  Failure  Rates 
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Figure  32.  Contract  Duration  Stated  Reasons  for  Failure 

4.  Contracts  Completed  by  Year 

The  years  were  categorized  based  on  their  completion  year  and  ranged  from 
2006  to  2013.  There  were  seven  contracts  in  2006,  18  in  2007,  51  in  2008,  94  in 
2009,  165  in  2010,  175  in  2011,  185  in  2012,  and  20  in  2013.  These  numbers  are 
clearly  shown  in  Table  16. 

Finding  7:  2007  had  the  highest  failure  rate  of  all  the  years  analyzed. 

There  was  one  2007  contract  labeled  a  failure,  giving  that  group  a  5.56% 
failure  rate  (see  Figure  33).  In  2007,  there  were  18  contracts  listed  in  PPIRS  for  the 
services  analyzed  (see  Table  16).  This  is  a  small  sample,  allowing  the  one  failure  to 
put  it  above  the  average  total  contract  failure  rate  of  3.08%.  The  contract  that  failed 
for  this  year  failed  for  its  business  relations  and  cost  control  (see  Figure  34).  The  first 
year  analyzed  was  2006  and  consisted  of  a  small  data  group.  There  were  only 
seven  contracts  in  the  data  from  this  year.  There  were  not  any  contracts  labeled  as 
failures,  giving  that  group  a  0%  failure  rate  (see  Figure  33).  There  were  51  contracts 
listed  in  PPIRS  for  the  2008-year  group.  There  were  two  labeled  failures  for  this 
year,  giving  2008  a  failure  rate  of  3.92%  (see  Figure  33).  This  rate  is  higher  than  the 
total  contract  average  of  3.08%.  The  two  failures  were  given  three  reasons  for  their 
failures  (see  Figure  34):  quality,  cost  control,  and  business  relations.  In  2009,  there 
were  94  contracts  entered  into  PPIRS.  There  were  five  labeled  failures,  giving  2009 
a  5.32%  failure  rate  (see  Figure  33).  The  2009  group’s  failure  rate  was  higher  than 
the  total  contract  failure  rate  of  3.08%.  The  failing  contracts  for  this  year  listed  four 
reasons  for  failing.  The  most  common  reason  for  failure  of  these  contracts  was 
quality  (see  Figure  34).  Three  other  reasons  were  listed:  business  relations, 
management  of  key  personnel,  and  utilization  of  small  business.  During  2010,  the 
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number  of  contracts  entered  into  PPIRS  grew  to  165  contracts  (see  Table  14).  There 
were  four  contracts  labeled  as  failures,  giving  the  year  a  2.42%  failure  rate  (see 
Figure  33).  This  year  was  under  the  total  contract  average  failure  rate  of  3.08%.  The 
most  common  reasons  for  failure  for  2010  were  quality  and  scheduling  (see  Figure 
34).  Although  not  the  most  common  reasons  for  failing,  there  were  three  other 
supporting  reasons  contracts  failed  during  this  year:  cost  control,  business  relations, 
and  management  of  key  personnel.  The  research  showed  175  contracts  inputted 
into  PPIRS  in  201 1 .  During  this  year,  there  were  five  labeled  failures,  giving  201 1  a 
failure  rate  of  2.86%  (see  Figure  33).  The  year  201 1  showed  that  its  failure  rate  was 
well  below  the  average  total  contract  failure  rate  of  3.08%.  The  most  common 
reason  for  failure  in  201 1  was  for  scheduling  problems  (see  Figure  34).  This  reason 
was  listed  for  all  five  of  the  failures.  There  were  other  reasons  for  failure,  such  as 
quality,  cost  control,  business  relations,  and  management  of  key  personnel.  In  2012, 
there  were  a  total  of  185  contracts,  which  included  four  labeled  failures,  giving  2012 
a  failure  rate  of  2.16%,  which  is  below  the  total  contract  average  of  3.08%  (see 
Figure  33).  The  most  common  reason  for  failure  of  these  contracts  was  for 
scheduling  (see  Figure  34).  Scheduling  failures  were  listed  for  two  of  the  contracts. 
The  other  reasons  for  failure  were  quality,  cost  control,  business  relations,  and 
management  of  key  personnel.  During  2013,  there  were  20  contracts  entered  into 
PPIRS.  There  was  one  labeled  a  failure,  giving  2013  a  failure  rate  of  5%  (see  Figure 
33).  This  failure  rate  is  higher  than  the  total  contract  average  of  3.08%.  The  reason 
for  the  failure  in  2013  was  cost  control  (see  Figure  34). 


Table  16.  Ending  Year  Total  Successes  and  Failures 


Ending  Year 
Total  Categories 

Total 

Successes 

Total 

Failures 

Total 

Contracts 

Failure 

Rates 

2006 

7 

0 

7 

0.00% 

2007 

17 

1 

18 

5.56% 

2008 

49 

2 

51 

3.92% 

2009 

89 

5 

94 

5.32% 

2010 

161 

4 

165 

2.42% 

2011 

170 

5 

175 

2.86% 

2012 

181 

4 

185 

2.16% 

2013 

19 

1 

20 

5.00% 

Total 

693 

22 

715 

3.08% 
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Figure  33.  Failure  Rate  by  Year 
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Figure  34.  Reasons  for  Failure  by  Year 
5.  Annual  Workload  Amount  in  Dollars 

Regional  MICC  office  annual  workload  in  dollars  spent  was  grouped  into  three 
categories.  The  categories  consisted  of  contract  totals  between  $0  and  $500  million, 
contracts  greater  than  $500  million  to  $1  billion,  and  contracts  greater  than  $1  billion. 
The  first  category  consisted  of  344  contracts,  the  second  category  had  256,  and  the 
final  category  contained  115  contracts.  These  figures  are  shown  in  Table  17. 
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Finding  8:  Regional  MICC  offices  that  had  spent  between  $0  and  $500 
million  in  annual  workload  had  the  highest  failure  rate. 

The  failure  rate  for  the  MICC  regions  between  $0  and  $500  million  in  annual 
workload  had  failure  rate  of  4.36%.  These  rates  are  shown  in  Figure  35.  The  primary 
reason  stated  for  failure  was  schedule  for  the  $0-$500  million  category.  The  second 
category  had  256  with  a  failure  rate  of  2.34%.  Management  of  key  personnel  was 
stated  as  the  reason  for  failures  for  category  two  and  the  final  category  contained 
115  contracts  with  only  one  of  them  being  labeled  a  failure,  giving  the  category  a 
0.87%  failure  rate.  The  final  category’s  only  labeled  failure  listed  cost  control  and 
business  relations  as  the  reason  for  failure.  These  results  are  shown  in  Figure  36. 


Table  17.  Annual  Workload  in  Dollars  Total  Success  and  Failures 


Contract  Dollar 
Amount 
Categories 

Total 

Successes 

Total 

Failures 

Total 

Contracts 

Failure 

Rates 

$0-$500M 

329 

15 

344 

4.36% 

>$500M-$1  B 

250 

6 

256 

2.34% 

>$1  B 

114 

1 

115 

0.87% 

Total 

693 

22 

715 

3.08% 

5.00% 


Annual  Contract  Workload  in  Dollars 


Figure  35.  Annual  Workload  in  Dollars  Failure  Rates 
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Figure  36.  Annual  Workload  in  Dollars  Stated  Reason  for  Failures  Among 

the  Ratings 

6.  Number  of  Actions  Completed 

The  number  of  actions  a  MICC  completed  in  a  year  was  separated  into  two 
different  categories.  The  first  category  ranged  from  zero  to  3,500  contracts.  The 
second  category  ranged  from  3,501  to  7,000  contracts.  These  categories  can  be 
seen  in  Table  18. 

Finding  9:  Contracts  managed  by  a  MICC  that  completed  3,501-7,000 
contracts  annually  had  the  highest  failure  rate  when  compared  to  MICCs  that 
completed  3,500  or  fewer  contracts  annually. 

There  were  277  contracts  in  this  category  in  the  database.  Of  these  277 
contracts,  nine  were  labeled  as  failures,  which  yields  a  failure  rate  of  3.25%  (see 
Figure  37).  Quality  and  the  management  of  key  personnel  were  each  referenced  five 
times  in  the  failed  contracts.  The  failure  rates  because  of  number  of  completed 
contracts  annually  are  shown  in  Figure  37.  Contracts  completed  in  a  MICC  that 
completed  3500  or  fewer  contracts  annually  had  the  next  highest  failure  rate  at 
2.91%.  There  were  413  of  these  contracts  in  the  database,  of  which  12  were  labeled 
as  failures.  Schedule  and  cost  control  were  each  referenced  five  times  while  quality 
was  referenced  four  times  in  the  failed  contracts.  There  was  no  data  available  for  25 
contracts,  of  which  one  was  labeled  a  failure.  The  rating  statistics  among  contract 
types  are  shown  in  Figure  38. 
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Table  18.  Number  of  Actions  Total  Success  and  Failures 


Number  of 
Actions 
Categories 

Total 

Successes 

Total 

Failures 

Total 

Contracts 

Failure 

Rates 

0-3500 

401 

12 

413 

2.91% 

3501-7000 

268 

9 

277 

3.25% 

Data  not  available 

24 

1 

25 

4.00% 

Total 

693 

22 

715 

3.08% 

Number  of  Completed  Contracts  Annually 


Figure  37.  Failure  Rates  by  Number  of  Completed  Contracts  Annually 
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Figure  38.  Number  of  Completed  Contracts  Annually  Stated  Reasons  for 

Failure 


7.  Authorized  1102  Billets 

There  were  147  contracts  that  were  completed  by  MICCs  that  had  between  0 
and  50  authorized  1102  billets,  543  contracts  that  had  over  50  authorized  1102 
billets,  and  25  contracts  with  no  data  available.  These  figures  are  shown  in  Table  19. 

Finding  10:  The  category  with  0-50  1102  billets  had  the  highest  failure 

rate. 


This  category  had  147  total  contracts  with  eight  failures,  giving  it  a  failure  rate 
of  5.44%  (see  Figure  39).  The  common  reason  for  these  failures  was  quality.  This 
reason  was  listed  for  five  of  the  eight-labeled  failed  contracts.  The  next  category  that 
contained  51-105  1102  authorized  billets  had  543  total  contracts.  This  category  had 
13  failures,  giving  this  group  a  2.39%  failure  rate.  The  most  common  reason  for 
failure  listed  for  this  group  was  schedule.  This  reason  was  listed  seven  times  out  of 
the  13  labeled  failed  contracts.  The  final  group  did  not  have  data  on  authorized 
billets.  This  group  contained  25  total  contracts  with  one  failure,  giving  this  category  a 
4%  failure  rate. 


Table  19.  Authorized  1102  Billets  Total  Success  and  Failures 


Authorized  1102 
Billets  Categories 

Total 

Successes 

Total 

Failures 

Total 

Contracts 

Failure 

Rates 

0-50 

139 

8 

147 

5.44% 

51-105 

530 

13 

543 

2.39% 

Data  not  available 

24 

1 

25 

4.00% 

Total 

693 

22 

715 

3.08% 
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Figure  39.  Failure  Rates  of  MICCs  by  1102  Billet  Authorizations 


Figure  40.  Reasons  for  Failures  by  Billet  Authorization 
8.  Percentage  of  1102  Billets  Filled 

To  analyze  how  1102  vacancies  affect  contract  success,  the  PPIRS  contract 
data  was  broken  into  six  groups.  These  groups  consisted  of  MICC  regions  that  had 
billets  50-60%  filled,  61-70%  filled,  71-80%  filled,  81-89%  filled,  and  90-100% 
filled;  and  there  was  contract  data  that  did  not  contain  adequate  billet  information. 
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Finding  11:  The  group  of  1102  billets  that  were  60-70%  filled  had  the 
highest  failure  rate  among  this  variable’s  categories. 

The  60-70%  group  had  a  4.94%  failure  rate  (see  Table  20  and  Figure  41). 
The  group  of  1 102  billets  that  were  61-70%  filled  had  81  total  contracts.  The  most 
common  reasons  listed  for  this  group  to  fail  were  schedule  and  cost  control  (see 
Figure  42).  The  group  of  1 102  billets  that  were  50-60%  filled  contained  data  for  22 
total  contracts.  This  group  had  one  labeled  a  failure,  which  gave  this  group  a  4.55% 
failure  rate.  This  rate  is  higher  than  the  total  contract  average  of  3.08%.  The  reason 
for  the  failure  in  this  group  was  quality,  scheduling,  and  management  of  key 
personnel.  The  next  group  of  1 102  billets  that  were  71%-80%  filled  consisted  of  122 
total  contracts.  There  were  five  contracts  labeled  as  failures,  which  gave  this  group  a 
4.1%  failure  rate.  This  group  was  higher  than  the  total  contract  failure  rate  of  3.08%. 
The  most  common  reasons  listed  for  the  failure  of  the  contracts  were  schedule  and 
cost  control.  The  group  that  contained  billets  filled  81%-89%  had  233  contracts  with 
five  labeled  a  failure.  The  five  failures  gave  this  group  a  2.15%  failure  rate.  This 
group  fell  below  the  total  contract  failure  rate  of  3.08%.  The  most  common  reasons 
for  failure  of  these  contracts  were  quality,  schedule,  and  business  relations.  The 
data  for  the  final  group  that  had  billets  90%-100%  filled  consisted  of  99  total 
contracts.  This  group  contained  one  labeled  a  failure,  giving  this  group  a  failure  rate 
of  1 .01%.  The  group  failure  rate  was  well  below  the  total  contracting  failure  rate  of 
3.08%.  The  reason  for  the  failure  in  this  group  was  management  of  key  personnel. 

Table  20.  Percentage  of  1102  Billets  Filled  Total  Success  and  Failures 


Percentage  of 
1102  Billet  Filled 
Categories 

Total 

Successes 

Total 

Failures 

Total 

Contracts 

Failure 

Rates 

50-60% 

21 

1 

22 

4.55% 

61-70% 

77 

4 

81 

4.94% 

71-80% 

117 

5 

122 

4.10% 

81-89% 

228 

5 

233 

2.15% 

90-100% 

98 

1 

99 

1.01% 

Data  not  available 

152 

6 

158 

0.00% 

Total 

693 

22 

715 

3.08% 
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Figure  41.  Failure  Rate  for  Different  1102  Billet  Vacancy  Groups 
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Figure  42.  Reasons  for  Failures  of  Different  1102  Vacancy  Groups 


E.  SIGNIFICANCE  TESTING 

The  research  results  were  analyzed  further  by  seeing  whether  the  actual 
failure  rates  are  significantly  different  from  what  would  be  expected  if  the  total 
contract  failure  rate  was  applied  to  each  variable.  The  Chi  Square  test  (Fisher’s 
exact  test)  was  used  to  test  the  different  variables.  The  null  hypothesis  for  this  test  is 
that  the  category  failure  rates  within  the  variables  are  not  significantly  different  than 
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the  total  contract  failure  rate  (3.08%).  The  null  hypothesis  is  rejected  if  the  p  value 
for  the  variable  is  less  than  0.05. 

This  test  showed  only  three  variables  that  had  categories  that  were 
significantly  different  when  the  average  total  contract  failure  rate  was  applied  to  the 
variable  and  category  (3.08%).  These  variables  were  contract  amount  (p  =  .036), 
contract  type  (p  =.009),  and  contracts  completed  by  MICC  region  (p  =.01 1 ).  The 
biggest  reason  for  a  significant  finding  for  contact  amount  was  the  category  of  $50 
million— $1  billion.  If  the  total  contract  failure  rate  were  applied  to  this  category,  it 
would  have  expected  to  fail  one  time,  but  in  actuality  it  had  eight  failures.  Within  the 
contract  type  variable,  three  contract  types  had  a  higher  difference  than  what  would 
have  been  expected.  Combination  type  contracts  (contracts  that  used  a  combination 
of  CPAF,  CPFF,  and  FFP  together)  was  expected  to  have  0.1  failures,  but  actually 
had  two.  FFP  type  contracts  were  expected  to  have  16,  but  had  14.  Other  type 
contracts  (such  as  labor  hours  and  time  and  materials)  was  expected  to  have  three 
failures,  but  had  only  one.  The  final  variable  that  showed  a  significant  difference  was 
contracts  by  MICC  region.  There  were  three  reasons  that  showed  a  significant 
difference  than  what  was  expected  and  the  actual  failures.  Fort  Eustis  was  expected 
to  have  7.3  failures,  but  only  had  one.  Fort  Knox  and  Fort  Hood  categories  were 
both  expected  to  have  3.5  failures,  but  had  six  failures  each. 


Table  21.  Chi  Square  and  Fisher’s  Exact  p  Value  Test  Results 


Contract  Variables 

p  value 

Significant? 

Type  of  Service  (RJSD) 

0.761 

No 

Contractual  Amounts 

0.036 

Yes 

Level  of  Competition 

1.00 

No 

Contract  Type 

0.009 

Yes 

MICC 

0.011 

Yes 

Duration 

0.567 

No 

End  Year 

0.619 

No 

Annual  Workload  in  Dollars 

0.142 

No 

Number  of  Completed  Actions 

0.709 

No 

Number  of  Authorized  1102  Billets 

0.098 

No 

Percent  of  1 102  billets  Filled 

0.301 

No 

The  remainder  of  the  variables  showed  a  p  value  greater  than  0.05  (see 
Table  21).  This  means  the  null  hypothesis  is  not  rejected.  Although  the  results  of  the 
different  categories’  failure  rates  fluctuated,  they  were  not  significantly  different  from 
the  total  contract  failure  rate.  These  fluctuations  could  be  from  acceptable 
randomness  within  the  data.  The  following  section  discusses  the  results  and 
managerial  implications  from  the  research  analysis  and  potential  areas  the  DoD  can 
focus  on  to  reduce  the  failure  rates  of  specific  service  contracts. 
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F. 


DISCUSSION  OF  RESULTS  AND  MANAGERIAL  IMPLICATIONS 


1.  Discussion  Overview 

The  purpose  of  this  research  is  to  identify  variables  in  the  service  contracting 
process  that  promote  successful  service  contracts.  This  section  examines  possible 
reasons  for  the  results  found  in  the  research.  It  also  considers  each  variable  the 
researchers  found  extraordinary  and  the  categories  of  these  variables.  Failed 
contracts  are  examined  by  looking  at  the  Service  Acquisition  Process  covered  in 
Chapter  II,  and  this  section  discusses  where  these  failures  could  be  addressed  to 
assist  government  contracting  agencies.  In  discussing  the  implications  of  the 
research,  it  had  to  be  assumed  that  the  contract  was  proper  (based  on  fair  and 
reasonable  price  and  accurately  reflecting  schedule  and  performance  requirements) 
in  terms  of  the  requiring  agencies’  needs.  This  allowed  proper  contract 
administration  to  discover  deficiencies  with  the  contractor’s  work  and  proper 
documentation  into  CPARS. 

•  Finding  1 :  The  S  type  contracts  had  the  highest  failure  rate  of  all 
the  product  service  codes  analyzed. 

The  first  variable  discussed  is  how  the  different  service  types  affected  the 
success  of  a  contract.  It  was  found  that  the  types  of  services  were  not  significantly 
different  from  the  overall  total  contract  failure  rate.  Even  though  contract  types  are 
not  shown  to  be  statistically  significant  in  terms  of  affecting  contract  success,  it  was 
observed  that  utilities  and  housekeeping  (S)  failed  six  times  for  business  relations 
and  six  times  for  management  of  key  personnel  out  of  the  1 1  reported  failures.  The 
business  relation’s  assessment  contains  the  assessment  of  the  contractors’  ability  to 
coordinate  their  business  activities,  such  as  their  attitude  towards  customers, 
customer  satisfaction,  and  their  cooperation  (DoD,  2011).  The  business  relation’s 
category  could  be  more  closely  addressed  during  the  planning  phase  of  the  service 
acquisition  process.  Step  three  of  this  process  is  market  research.  One  possible  way 
of  mitigating  future  failures  of  this  type  is  to  ensure  that  the  contracting  agency  gives 
this  step  extra  attention.  This  step  contains  seven  sub-elements  that  ensure  that  the 
acquisition  team  has  a  detailed  understanding  of  current  market  practices  in  the  area 
of  the  service  being  provided.  This  will  assist  with  making  performance  requirements 
more  attainable  for  the  contractor  (DoD,  2012).  The  next  common  failure  for  utilities 
and  housekeeping  services  was  the  management  of  key  personnel.  This 
assessment  area  measures  the  contractor’s  ability  to  maintain  qualified  individuals  in 
key  positions  as  outlined  in  the  contract  (DoD,  2011).  These  numbers  suggest  that 
the  contracting  agency  should  focus  on  the  execution  phase  of  the  process  and  give 
extra  attention  to  the  execute-strategy  step.  This  phase  and  step  goes  from  the 
issuance  of  the  RFP  to  the  post-award  implementation/transition.  Within  this  step  is 
the  source  selection.  Prior  to  conducting  the  source  selection  process,  the 
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acquisition  team  needs  to  ensure  that  the  source  selection  evaluators  (SSE)  are 
properly  trained.  The  team  must  also  ensure  that  the  SSEs  understand  how  to 
properly  evaluate  the  offeror’s  management  proposal.  This  proposal  gives  the 
offeror’s  breakdown  of  key  personnel  that  will  be  involved  in  the  contract. 

•  Finding  2:  The  contract  grouping  that  consisted  of  contracts 
worth  a  dollar  amount  greater  than  $50  million-$1  billion  had  the 
highest  failure  rate  of  all  the  groups  in  the  analysis. 

As  discussed  in  the  previous  section,  this  variable  proved  to  have  a  significant 
impact  on  service  contracting  success.  The  category  within  this  variable  appearing 
to  be  the  most  significant  was  the  category  containing  contracts  greater  than  $50 
million— $1  billion.  This  group  had  the  highest  failure  rate  at  8.7%.  The  most  common 
reason  for  this  failure  was  cost  control.  The  cost  control  rating  contains  an 
assessment  of  the  contractors’  ability  to  “forecast,  manage,  and  control  the  cost” 
(DoD,  201 1 ,  p.  A3-7)  associated  with  conducting  their  services.  Again,  this  research 
assumes  that  the  contractor  was  provided  a  proper  contract,  which  in  turn  allowed 
proper  contract  administration  that  resulted  in  an  identified  cost  control  failure 
documented  in  CPARS.  This  leads  one  to  assume  that  a  contract  was  awarded  to  a 
business  that  was  not  able  to  perform  the  cost  control  function  sufficiently.  This 
leads  back  to  the  execute  phase  and  execute  strategy  step  needing  extra  attention 
to  mitigate  future  failures.  The  acquisition  team  needs  to  ensure  that  all  SSEs  are 
properly  trained  on  how  to  accurately  evaluate  the  cost  proposals  to  ensure  the 
business  fully  understands  the  contract  requirements.  If  the  cost  proposal  is  not 
properly  evaluated,  a  contractor  may  be  awarded  the  contract  that  does  not  fully 
grasp  the  contract  requirements  leading  to  future  cost  overruns.  Another  category 
within  this  data  was  worth  analyzing:  The  group  that  contained  contract  dollar 
amounts  greater  than  $1  million— $1 0  million.  Although  this  category’s  failure  rate 
was  actually  one  of  the  lowest  failure  rates  (2.15%),  the  reason  for  its  failure  appears 
to  be  important  for  analysis.  This  category  had  a  total  of  10  failures.  Of  these  10 
failures,  seven  were  listed  for  quality  reasons.  The  quality  assessment  factor 
contains  an  evaluation  of  the  contractor’s  qualitative  performance  and  compares  it  to 
the  requirements  stated  in  the  contract  (DoD,  2011).  Because  these  contract  failures 
were  proper  contracts  allowing  proper  documentation  into  CPARS,  the  acquisition 
team  needs  to  ensure  they  focus  on  training  of  their  SSEs.  Acquisition  teams  that 
are  a  part  of  contracts  within  this  dollar  amount  need  to  focus  the  execute  phase  and 
execute  strategy  step  of  the  services  acquisition  process.  They  need  to  ensure  that 
their  SSEs  are  properly  trained  and  can  evaluate  the  contractor’s  technical  proposal. 
This  proposal  will  help  the  acquisition  team’s  understanding  of  the  contractor’s  plan 
to  complete  the  quality  requirements  in  the  contract. 
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•  Finding  3:  Contracts  competed  competitively  had  the  highest 
failure  rate  when  compared  to  the  other  two  forms  of  competition 
available. 

Competitively  competed  contracts  had  the  highest  of  the  category’s  failure 
rates  at  3.15%.  These  contracts  most  often  failed  because  of  quality  and  schedule. 
Although  out  of  523  contracts,  only  17  failed,  this  gave  the  category  a  slightly  higher 
failure  rate  than  the  total  contract  failure  rate  of  3.08%.  This  category  failed  seven 
times  for  scheduling  reasons  and  seven  times  for  cost  control.  The  numbers  in  these 
two  areas  suggest  the  execute  phase  and  the  execute  strategy  step  in  the  service 
acquisition  process  needs  to  be  scrutinized.  The  acquisition  team  needs  to  ensure 
proper  training  of  the  SSEs  on  how  to  properly  evaluate  the  contractor’s  cost 
proposal  and  technical  proposal.  These  proposals  show  sufficient  understanding  to 
successfully  fulfill  the  contract’s  requirements.  If  the  cost  proposal  is  not  adequately 
evaluated,  this  may  result  in  future  cost  overruns.  Inadequate  evaluation  of  the 
contractor’s  technical  proposal  could  result  in  the  contractor’s  inability  to  meet 
scheduling  requirements. 

•  Finding  4:  Contracts  structured  as  a  combination  contract  had  the 
highest  failure  rate  when  compared  to  the  other  five  types  of 
available  contracts. 

The  contract  type  variable  was  significant  on  the  outcome  of  the  service 
contract.  There  are  three  categories  worth  mentioning  in  this  variable.  They  are  the 
combination,  CPAF,  and  CPIF.  The  combination  contract  type  showed  a  failure  rate 
of  50%.  This  could  be  because  of  the  relatively  low  number  of  this  contract  type  in 
the  data.  The  data  consisted  of  only  four  contracts  and  two  of  these  showed  failures. 
The  other  two  categories  (CPAF,  5.17%;  CPFF,  5.56%)  showed  a  failure  rate  nearly 
double  that  of  the  next  lowest  category  (FFP,  2.67%).  Contracting  officers  using 
these  types  of  contract  vehicles  for  their  procurement  of  services  should  be  aware  of 
these  failure  rates.  The  leading  and  reccurring  theme  among  all  three  of  these 
categories  was  cost  control.  The  cost  type  contracts  places  more  financial 
responsibility  on  the  government.  These  contract  vehicles  should  be  avoided  when 
the  requirements  of  the  service  allow  for  a  FFP  contract.  To  gain  better  knowledge 
and  build  better  requirements  to  allow  for  a  FFP  contract,  Step  2:  Review  Current 
Strategy  of  the  Planning  Phase  could  help  mitigate  the  use  of  cost-based  contracts. 
This  step  involves  analyzing  requirements  and  desired  results  of  current  ongoing 
service  contracts  and  attempts  to  draw  parallels  to  gather  necessary  information  to 
develop  exact  requirements  needed  by  the  service.  Thorough  execution  of  Step  3: 
Market  Research  in  the  Planning  Phase  could  also  assist  with  moving  away  from  the 
use  of  these  cost-based  contracts  and  place  more  cost  responsibility  on  the 
contractor.  Market  research  entails  gaining  a  detailed  understanding  of  what  the 
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marketplace  associated  with  the  service  can  provide.  This  understanding  assists  the 
acquisition  team  and  avoids  wasting  taxpayer  dollars  (DoD,  2012).  The  acquisition 
team  needs  to  examine  the  execute  strategy  in  the  execute  phase  and  ensure 
proper  training  of  its  SSEs.  The  cost  proposals  of  these  cost  contracts  may  not  have 
been  evaluated  correctly.  Without  proper  evaluation,  a  contractor  without  adequate 
understanding  of  the  requirements  may  be  awarded  the  contract.  This  will  lead  to  a 
greater  possibility  of  cost  overruns. 

•  Finding  5:  Contracts  completed  by  Fort  Hood  had  the  highest 
failure  rate  compared  to  the  other  four  MICC  regions. 

The  MICC  region  variable  was  examined  and  was  shown  to  have  a  significant 
effect  on  the  success  of  service  contracts.  There  were  three  reasons  that  showed  a 
significant  difference  in  the  expected  category  failures  and  the  actual  failures.  Fort 
Hood  and  Fort  Knox  regional  categories  both  were  expected  to  have  3.5  failures  but 
actually  had  six  failures.  Fort  Hood  had  the  highest  failure  rates,  and  at  first  glance, 
this  finding  may  appear  to  be  negative.  This  finding  may  have  resulted  from  the 
proper  use  and  documentation  within  the  CPARS  system,  thus  showing  a  more 
realistic  outcome  of  contracting  results.  This  MICC  region  may  have  more  CORs 
assigned  to  contracts  under  their  control  allowing  for  better  oversight  and  assessing 
contract  failure  when  appropriate.  The  most  cited  reason  for  failure  in  this  region  was 
for  quality.  Acquisition  teams  within  this  region  may  want  to  examine  how  the 
technical  proposals  are  being  evaluated  in  the  execution  strategy  step.  They  need  to 
ensure  their  SSEs  are  properly  trained  to  mitigate  future  failures  for  quality  reasons. 
Fort  Knox  did  not  have  a  pattern  for  reasons  of  failure.  They  were  fairly  evenly 
distributed  and  did  not  show  a  possible  weakness  in  the  process  to  be  improved. 

The  next  MICC  region  that  appeared  significant  was  Fort  Eustis.  The  Fort  Eustis 
region  fell  significantly  below  the  expected  failure  rate.  This  finding  also  may  be 
because  of  manning  shortages  or  training  deficiencies.  This  region  may  not  have  the 
CORs  to  enforce  proper  contract  administration  and  show  deficient  past 
performance  records  through  CPARS. 

•  Finding  6:  Contracts  with  a  duration  between  0-90  days  had  the 
highest  failure  rate  compared  to  the  other  four  contact  durations 
analyzed. 

As  discussed  in  the  previous  section,  contract  duration  was  found  not  to  have 
any  significantly  different  failure  rates  than  the  total  contract  failure  rate.  The  first 
category  of  contracts  with  a  duration  of  0-90  days  had  a  failure  rate  of  7.41%.  The 
small  sample  size  explains  this  high  failure  rate.  The  271 -365-day  contractual 
duration  had  failures  for  all  the  reasons  listed  that  was  almost  evenly  distributed.  The 
data  did  not  show  that  duration  was  meaningful  to  the  success  of  a  service  contract. 
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•  Finding  7:  2007  had  the  highest  failure  rate  of  all  the  years 
analyzed. 

After  the  significance  test  was  completed  on  this  variable,  it  showed  that  it 
was  not  significantly  different  than  the  total  contract  failure  rate.  Although  not 
significant,  the  year  2007  had  18  contracts  with  one  failure,  giving  it  the  highest 
failure  rate  at  5.56%.  It  failed  one  time  for  cost  control  and  business  relations.  This 
high  failure  rate  appears  to  be  the  result  of  a  small  sample  size.  The  year  201 1  had 
a  lower  failure  rate  than  most  of  the  other  years  included  in  the  data.  The  data 
showed  201 1  had  five  failed  contracts.  All  of  these  failures  also  had  schedule  listed 
as  a  failed  evaluation  criteria.  This  criterion  is  an  assessment  of  the  ability  of  the 
contractor  to  meet  schedules  outlined  in  the  contract  such  as  “task  orders, 
milestones,  schedules,  and  administrative  requirements”  (DoD,  2012,  p.A3-7). 

These  failures  may  have  been  mitigated  through  a  more  thorough  contracting 
process  such  as  conducting  a  more  deliberate  execution  phase.  This  phase  consist 
of  the  pre-solicitation  conference  giving  the  contractor  and  agency  requiring  the 
service  a  more  common  understanding  of  expectations  and  requirements  (Apte  et 
al. ,  2006).  These  findings  may  also  show  that  in  201 1  the  planning  phase  and 
development  phase  was  conducted  more  thoroughly  than  in  previous  years,  allowing 
the  agencies  to  better  conduct  contract  administration  and  hold  the  contractors 
responsible  for  scheduling  deficiencies  and  documenting  these  deficiencies  in 
CPARS.  The  2012  and  2013  year  groups  show  a  decline  in  scheduling  failures, 
allowing  one  to  assume  agencies  and  contractors  were  able  to  come  to  a  better 
common  understanding  of  scheduling  requirements. 

•  Finding  8:  Regional  MICC  offices  that  had  spent  between  0  and 
$500  million  in  annual  workload  had  the  highest  failure  rate. 

Regional  MICC  offices  that  did  between  $0-$500  million  in  annual  workload 
had  the  highest  failure  rate  at  4.36%.  Its  most  common  reasons  for  failure  were 
schedule  and  quality.  To  mitigate  future  failures  in  the  same  areas,  acquisition  teams 
should  focus  on  the  execution  phase  and  execution  strategy  step  within  the  service 
acquisition  process.  This  step  is  the  source  selection  for  the  contract.  The 
acquisition  team  needs  to  ensure  proper  training  of  its  SSEs  to  ensure  a  good 
evaluation  of  the  technical  proposal  within  the  contractor’s  offer.  Without  proper 
training  on  how  to  evaluate  these  proposals,  the  government  is  at  risk  of  awarding  a 
contract  to  a  contractor  that  is  unable  to  comply  with  the  contract  scheduling  and 
quality  requirements.  Although  the  different  annual  workload  failure  rates  did  not 
show  that  they  were  statistically  different  than  the  total  contract  failure  rate,  the 
numbers  did  appear  to  have  an  overarching  pattern.  As  the  annual  workload 
increased,  the  failure  rate  decreased.  The  Chi  Square  test  shows  that  these 
differences  could  be  because  of  randomness,  and  how  the  data  was  gathered  for 
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analysis,  there  does  appear  to  be  a  trend  within  this  data.  These  findings  appear  to 
be  counterintuitive  to  what  one  might  expect.  This  trend  could  be  explained  through 
how  the  MICCs  were  staffed.  These  MICC  offices  may  have  been  projected  to  have 
greater  dollar  amounts  passing  through  their  regions  because  of  deployments  and 
higher  operational  tempos  in  their  areas.  Given  this  projection,  the  MICC  offices  may 
have  been  prioritized  and  manned  with  more  personnel.  This  would  allow  for  more 
people  to  conduct  the  service  acquisition  process  more  thoroughly  and  with  greater 
success.  This  greater  workload  would  also  allow  for  a  higher  learning  curve  and 
allow  the  acquisition  teams  within  these  regions  to  better  understand  their  jobs  and 
responsibilities.  A  counterargument  to  this  could  be  that  these  higher  workloads  do 
not  allow  for  the  execution  phase  to  be  properly  conducted.  This  allows  for  less  than 
proper  contract  administration  and  failed  contracts  not  properly  documented  on 
CPARS. 


•  Finding  9:  Contracts  managed  by  a  MICC  that  completed  3,501- 
7,000  contracts  annually  had  the  highest  failure  rate  when 
compared  to  MICCs  that  completed  3,500  or  fewer  contracts 
annually. 

The  MICC  regions  that  consisted  of  contracting  actions  between  3,501  and 
7,000  contracts  annually  had  the  highest  failure  rates.  This  category  failed  most 
commonly  for  quality  and  management  of  key  personnel.  These  failures  in  CPARS 
suggest  the  MICCs  that  fall  within  these  categories  need  to  look  at  how  they  are 
executing  their  source  selection  within  the  execution  strategy  step.  The  MICCs  need 
to  ensure  the  acquisition  teams  are  properly  training  their  SSEs  on  how  to  evaluate 
the  contractor’s  management  and  technical  proposals.  Acceptance  of  a  contractor’s 
proposal  without  proper  understanding  of  the  contract  requirements  raises  the  risk  of 
contract  failure.  As  discussed  previously,  this  variable  had  p  value  greater  than  0.05, 
showing  that  its  categories’  failure  rates  were  not  significantly  different  from  what 
would  have  been  expected  given  the  total  contract  failure  rate.  Although  not 
significantly  different,  the  numbers  do  appear  to  reflect  a  trend  that  as  the  workload 
increases,  the  failure  rate  also  increases.  Although  the  Chi  Square  test  shows  that 
these  differences  may  be  from  randomness  within  the  data  population,  this  trend 
does  allow  the  researchers  to  speculate.  It  appears  that  as  the  contracting  workload 
increases,  the  number  of  trained  personnel  for  government  contracting  is  not  being 
adequately  increased.  GAO  (2013b)  and  Gansler  (2011)  have  both  identified  that 
without  the  proper  trained  personnel  to  conduct  the  contracting  process,  the 
government  runs  the  risk  of  contract  failure.  As  discussed  in  Chapter  I,  history  has 
shown  dramatic  increase  in  service  contracting  over  the  past  21  years  (Ellman  et  al., 
2011).  With  this  increase,  it  is  important  to  increase  the  proper  workforce  as  well. 
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•  Finding  10:  The  category  with  0-50  1102  billets  had  the  highest 
failure  rate. 

MICC  regions  that  had  0-50  1102  billets  had  the  highest  failure  rate  at  5.44%. 
Its  most  common  reason  for  failing  was  quality.  These  numbers  may  suggest  the 
acquisition  team  within  these  categories  need  to  focus  their  attention  on  the 
execution  phase.  Within  this  phase  the  teams  need  to  examine  how  they  are  training 
their  SSEs.  These  numbers  may  suggest  they  are  not  awarding  the  contract  to  the 
best  contractor  or  their  requirements  for  this  area  are  not  attainable  by  the 
contractor.  Although  this  variable  was  shown  not  to  be  statistically  significant,  the 
results  of  the  research  did  appear  to  have  an  obvious  trend.  This  data  showed  as 
the  authorized  1102  billets  increased,  the  failure  rate  decreased.  Although  the  Chi 
Square  test  p  value  states  these  results  may  be  from  randomness,  it  appears  to  be  a 
result  one  would  expect  to  find.  One  can  speculate  that  MICC  offices  with  higher 
authorized  1102  billets  also  has  more  1102  specialists  in  the  position,  whether  all  the 
positions  are  filled  or  not.  The  results  of  this  research  support  the  GAO  (2013b)  and 
report  Gansler  (2011)  published  stating,  without  properly  trained  personnel  to 
conduct  the  contract  process  the  government  runs  the  risk  of  contract  failure. 

•  Finding  11:  The  group  of  1102  billets  that  were  60-70%  filled  had 
the  highest  failure  rate  among  this  variable’s  categories. 

The  1102  billet  percentage  category  that  showed  60-70%  filled  had  the 
highest  failure  rate  at  4.94%.  It  showed  to  fail  most  often  because  of  schedule  and 
cost  control.  These  numbers  suggest  that  MICCs  within  these  fill  rates  need  to  focus 
on  the  execution  strategy  step  to  mitigate  their  risk  of  future  failures  in  the  same 
areas.  The  MICCs  need  to  look  at  how  they  are  training  their  SSEs  in  the  areas  of 
evaluating  the  cost  and  technical  proposals.  These  two  proposals  lay  out  how  the 
contractor  plans  to  fulfill  the  contracts  requirements  in  forecasting  cost  and  meeting 
scheduling  requirements.  These  proposals  show  whether  the  contractor  really 
understands  the  contract  requirements.  If  the  contract  is  awarded  to  a  contractor  that 
does  not  understand  the  contract  requirements,  then  the  government  is  at  higher  risk 
for  cost  overruns  and  schedule  delays.  This  variable’s  failure  rates  again  proved  to 
be  not  significantly  different  than  the  total  contract  failure  rate.  The  Chi  Square  test 
showed  that  these  differences  in  the  variables  failure  rates  could  be  due  to 
randomness  or  how  the  data  was  gathered.  The  results  of  this  analysis  had  a 
straightforward  trend  that  is  hard  to  ignore.  The  trend  in  this  analysis  showed  as  the 
percentage  for  billets  filled  increased,  the  failure  rate  decreased.  This  trend  again 
supports  the  GAO  (2013b)  and  Gansler  (2011)  claim  of  the  importance  of  having 
trained  personnel  in  the  positions  for  proper  contracting  process  phases.  Another 
trend  identified  in  this  variable  was  that  the  cost  control  failures  decreased  as  the 
billets  are  filled  to  more  appropriate  levels.  The  direction  of  these  numbers  show,  as 
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there  are  less  trained  personnel,  the  government  does  run  the  risk  of  paying  too 
much  and  higher  risk  of  contract  failure. 

G.  SUMMARY 

This  chapter  began  by  reviewing  an  overview  of  the  data.  This  chapter  then 
analyzed  the  data  to  answer  the  research  questions.  The  chapter  covered  how  the 
research  used  the  Chi  Square  test  on  the  data.  This  showed  if  the  average  total 
contract  failure  rate  was  applied  across  the  categories  of  the  variables  whether  there 
would  be  a  significant  difference  from  what  was  actually  captured  within  the  data. 
The  final  section  in  this  chapter  was  a  discussion  of  results  and  managerial 
implications.  This  section  discussed  in  further  detail  what  was  found  in  the  analysis 
of  the  variables’  failure  rates  and  categories’  reasons  for  failures.  Chapter  V  covers 
the  summary  of  the  research,  conclusion  of  the  findings,  and  areas  for  further 
research  to  enhance  the  knowledge  of  service  contracting. 
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V.  SUMMARY,  CONCLUSION,  AND  AREAS  FOR  FURTHER 

RESEARCH 


A.  SUMMARY 

Over  the  last  few  decades,  services  contracting  has  continued  to  grow. 
Contracting  for  services  has  grown  in  relation  to  product  contracting  over  the  last  21 
years,  and  was  the  fastest  growing  area  for  DoD  contract  spending  at  a  growth  rate 
of  6.1%  (Ellman  et  al.,  2011).  This  growth  in  dollars  spent  has  brought  increased 
political  attention  and  scrutiny. 

Contract  management  has  been  listed  on  the  GAO’s  High  Risk  Series  since 
1992  (GAO,  2013a).  Problems  in  contract  administration  have  also  resulted  in  142 
reports  completed  by  the  DoD  IG  in  five  years  (Burton,  2009).  Gansler  (201 1 ) 
identified  that  most  current  regulations,  policies,  training,  standards,  education,  and 
management  structures  re  set  up  to  optimize  product  acquisition,  and  only  a  few 
focused  on  the  optimization  of  service  acquisition. 

The  DoD  has  responded  to  these  problems  to  improve  service  acquisition  in 
several  different  ways.  USD(AT&L)  Frank  Kendall  released  BBP  2.0  to  place  senior 
leadership  to  manage  service  acquisitions.  The  DoD  also  forced  the  components  to 
adopt  a  standardization  of  taxonomy  in  order  to  better  classify  acquisitions  (GAO, 
2013b). 

With  these  improvements,  service  acquisitions  still  have  problems.  The 
problems  could  be  from  a  lack  of  standard  definition  for  success.  With  the  differing 
goals  and  objectives  of  the  different  stakeholders,  a  proxy  definition  of  success  is 
needed  to  measure  success.  Because  contract  success  and  failure  is  recorded 
through  CPARS,  this  information  is  used  for  the  proxy  definition  for  success. 

B.  CONCLUSION 


1.  Research  Findings 

The  purpose  of  this  research  was  to  identify  variables  in  the  service 
contracting  process  that  promote  successful  service  contracts.  To  identify  these 
variables,  the  research  was  focused  on  answering  these  four  research  questions: 

•  Do  the  types  of  services  being  acquired  affect  the  success  of  a  service 
contract? 

•  Do  the  contractual  amounts  affect  the  success  of  a  service  contract? 

•  Does  the  level  of  competition  used  affect  the  success  of  a  service 
contract? 
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•  Does  the  contract  type  affect  the  success  of  a  services  contract? 

In  answering  the  first  research  question,  it  was  discovered  that  the  types  of 
services  being  acquired  is  not  a  significant  influencer  on  the  success  of  a  service 
contract  (p  =  .761).  The  category  of  S-type  services  (utilities  and  housekeeping)  did 
have  the  highest  failure  rate  at  3.77%  and  is  closely  followed  by  J-type  services 
(maintenance,  repair,  and  rebuilding  of  equipment)  at  3.45%,  R-type  services 
(professional  administrative  and  management  support)  at  2.72%,  and  D-type 
services  (automatic  data  processing  and  telecommunications)  at  0%.  Though  the 
failure  rates  were  different  across  the  service  types  this  research  examined,  the 
overall  failure  rate  was  close  to  the  rate  expected  for  the  types  of  services  examined 
in  this  research. 

In  answering  the  second  research  question,  the  data  indicated  that  a 
statistically  significant  relationship  existed  between  contractual  amounts  and  the 
success  of  a  service  contract  (p  =  .036).  Contracts  ranging  from  $50  million  to  $1 
billion  had  a  failure  rate  over  three  times  higher  than  the  lower  dollar  categories 
(8.70%  vs.  2.86%).  It  was  also  found  that  the  single  highest  point  of  failure  in  the 
failing  contracts  was  cost  control  at  six  references  out  of  the  eight  failures.  Cost 
control  was  noted  as  a  source  of  failure  six  times,  which  was  twice  as  often  as  the 
next  most  common  source  of  failure  which  was  schedule  at  three  times. 

In  answering  the  third  research  question,  the  data  indicated  that  there  was  not 
a  statistically  significant  relationship  between  the  level  of  competition  used  and  the 
success  of  a  service  contract  (p  =  1 .00).  Competitive  and  non-competitive  contracts 
did  have  failure  rates  at  3.15%  and  2.91%,  respectively.  Though  the  failure  rates 
were  different  between  the  level  of  competition  used,  the  overall  failure  rate  was 
close  to  the  rate  expected  for  the  types  of  services  examined  in  this  research. 

Lastly,  in  answering  the  fourth  research  question,  the  data  indicated  that  there 
is  a  statistically  significant  relationship  between  the  contract  type  and  the  success  of 
a  service  contract  (p  =  .009).  Contracts  that  used  a  combination  of  CPFF,  FFP,  and 
CPAF  had  the  highest  failure  rate  in  the  database  at  50%,  although  it  must  be  stated 
that  the  sample  size  of  combination  contracts  was  over  130  times  smaller  than  the 
largest  population  in  the  same  category  (FFP,  522  entries).  The  stated  reason  the 
contracts  were  labeled  as  a  failure  in  all  of  the  combination  contracts  was  schedule 
and  cost  control.  The  data  also  shows  that  the  Army  uses  FFP  contracts  for  services 
over  five  times  more  often  than  the  next  most  commonly  used  contract  type.  FFP 
contracts  also  have  the  third  lowest  failure  rate,  which  indicates  that  FFP  contracts 
should  continue  to  be  used  as  often  as  possible  because  of  their  historically  low 
failure  rate.  The  results  to  the  research  questions  can  also  be  seen  in  Table  22. 
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Table  22.  Contract  Variables  Effect  on  Success  of  a  Service  Contract 


Contract  Variables 

Effect  on 
Success  of  a 
Service  Contract 

Type  of  Service  (RJSD) 

No 

Contractual  Amounts 

Yes 

Level  of  Competition 

No 

Contract  Type 

Yes 

2.  Additional  Findings 

While  compiling  the  database,  several  additional  variables  were  analyzed  that 
yielded  some  interesting  results  regarding  variables  that  influence  the  success  of 
service  contracts.  These  additional  variables  were  failure  rates  by  MICC,  duration  of 
contracts,  contracts  by  fiscal  year,  annual  contract  workload  in  dollars,  and  the 
percent  fill  of  authorized  1102  billets  by  MICC. 

Of  all  these  additional  variables  analyzed,  only  the  failure  rates  of  the 
contracts  completed  by  MICCs  were  statistically  significant  (p  =  .011).  The  data 
clearly  showed  that  Fort  Eustis  had  an  overwhelmingly  lower  failure  rate  (0.42%)  for 
service  contracts  than  the  other  four  MICCs  that  were  analyzed.  Fort  Eustis  had  a 
failure  rate  over  four  times  lower  than  the  next  lowest  failure  rate  (Fort  Bragg  was 
1.82%)  and  over  12  times  lower  than  the  highest  failure  rate  (Fort  Hood  was  5.26%). 
Additional  analysis  is  needed  to  determine  what  caused  these  failure  rates  to  be 
different  among  the  MICCs. 


3.  Recommendations 

Even  with  the  low  failure  rate,  the  data  showed  three  statistically  significant 
relationships.  Contractual  amounts,  contract  types,  and  MICCs  all  had  a  p  value  of 
less  than  0.05.  Contracts  valued  at  $50  million  up  to  $1  billion  had  the  highest  failure 
rate  at  8.70%.  Cost  control  was  referenced  six  times  as  the  reason  a  contract  was 
labeled  as  a  failure,  which  suggests  that  contracts  in  this  category  need  better 
procurement  planning  during  the  contracting  process,  as  discussed  in  Chapter  IV. 
Contract  types  that  used  a  combination  type  contract  had  the  largest  failure  rate  of 
any  of  the  other  contract  types  at  50%.  CPIF  and  CPAF  also  had  high  failure  rates. 
This  suggests  that  these  contracts  needed  better  procurement  planning  and 
solicitation  planning.  These  two  contracting  phases  is  where  the  requirements  of  the 
services  are  examined  and  contract  vehicles  chosen.  The  SOW  and  project  scope 
needs  to  be  developed  and  matured  enough  to  allow  for  FFP  contract  to  be 
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accepted  by  the  contractor  in  both  these  areas.  This  high  failure  rate  also  suggests 
that  using  a  combination  type  contract  should  be  avoided,  since  it  has  a  high 
likelihood  of  failure.  Fort  Eustis  had  the  lowest  failure  rate  of  any  of  the  MICCs 
examined  in  this  research  at  0.42%;  as  seen  in  Chapter  IV,  this  could  be  because  of 
a  lack  of  trained  personnel  to  adequately  enforce  proper  contract  administration. 

This  could  also  be  from  proper  manning  with  adequately  trained  personnel  allowing 
for  adequate  contract  management. 

C.  AREAS  FOR  FURTHER  RESEARCH 

Additional  research  would  be  valuable  in  several  areas  to  further  define  the 
management  levers  that  drive  success  in  service  contracts.  The  database  used  in 
this  research  focused  heavily  on  outcome  successes  defined  by  Hagan  et  al.  (2012). 
As  such,  expanding  this  research  to  include  the  DAWIA  training  profile,  numbers  of 
assigned  personnel,  and  the  number  of  CORs  the  MICCs  use  would  add  significant 
insight  to  this  research.  This  research  could  also  be  expanded  to  include  other  DoD 
organizations  because  the  contracts  analyzed  in  this  report  are  only  Army  service 
contracts. 

Additional  research  would  also  be  useful  in  analyzing  a  larger  number  of 
contracts  that  received  either  an  unsatisfactory  or  marginal  rating.  Examining  all  the 
contracts  in  the  PPIRS-RC  database  that  receive  a  failure  label  might  yield  useful 
results  when  combined  with  this  study. 

While  conducting  preparation  for  this  research,  the  researchers  found  a  large 
amount  of  information  on  how  government  contracting  has  historically  been 
unsuccessful.  This,  along  with  the  GAO’s  placement  of  contract  management  on  the 
High  Risk  list  since  1992,  causes  the  expectation  of  a  high  service  contract  failure 
rate.  This  data  contained  715  contracts  with  only  22  total  contract  failures.  This  set 
the  total  contract  failure  rate  at  3.08%.  This  suggests  that  additional  analysis  is 
needed  to  determine  whether  the  CPARS  database  is  accurately  capturing  the 
performance  of  a  service  contract.  The  results  found  in  this  research  are  all  based 
on  CPARS  entries,  which  could  be  biased  because  of  a  conflict  of  interest.  This 
conflict  of  interest  is  created  since  whatever  rating  the  AO  gives  to  a  contract  may  be 
a  reflection  in  the  way  that  person  structured  or  managed  the  contract.  This  could 
potentially  cause  the  rating  a  contractor  receives  to  be  higher  than  it  should  have 
actually  received.  The  findings  in  this  report,  combined  with  a  service  quality  gap 
analysis,  would  add  valuable  insight  into  what  makes  service  contracts  successful. 
The  service  quality  gap  analysis  would  reveal  the  accuracy  of  the  rating  a  contact 
received  in  CPARS,  and  would  be  useful  in  determining  whether  CPARS  accurately 
captures  a  contracts  performance. 
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APPENDIX:  CPAR  EVALUATION  RATINGS  AND  DEFINITIONS 

(DOD,  2011) 


EjvaluatiornLating^kcfinition^^ixcludinj^^iliziiliorMjnjirnainJuxiiicss^ 


Katin” 

Definition 

Note 

Exceptional 

Performance  meets  contractual 
requirements  and  exceeds  many  to  the 
Government's  benefit.  The  contractual 
performance  of  the  element  or  sub- 
element  being  assessed  was  accomplished 
with  few  minor  problems  for  which 
corrective  actions  taken  by  the  contractor 
was  highly  effective. 

To  justify  an  Exceptional  rating,  identify’  multiple 
significant  events  and  state  how  they  were  of  benefit 
to  tlx:  Government  A  singular  benefit  however, 
could  be  of  such  magnitude  that  it  alone  constitutes 
an  Exceptional  rating.  Also,  there  should  have  been 
NO  significant  weaknesses  identified. 

Very  Good 

Performance  meets  contractual 
requirements  and  exceeds  some  to  the 
Government's  benefit.  The  contractual 
performance  of  the  element  or  sub- 
clcmcnt  being  assessed  w  as  accomplished 
with  some  minor  problems  for  which 
corrective  actions  taken  by  the 
contractor  was  effective. 

To  justify  a  Very  Good  rating,  identify  a  significant 
event  and  state  how  it  was  a  benefit  to  the 

Government.  There  should  have  been  no  significant 
weaknesses  identified. 

Satisfactory 

Pcrfbnuancc  meets  contractual 
requirements.  The  contractual 
performance  of  the  element  or  sub- 
element  contains  some  minor  problems 
for  which  corrective  actions  taken  by  the 
contractor  appear  or  were  satisfactory. 

To  justify  a  Satisfactory  rating,  there  should  have 
been  only  minor  problems,  or  major  problems  the 
contractor  recovered  from  without  impact  to  the 
contract  There  should  have  been  NO  significant 
weaknesses  identified.  A  fundamental  principle  of 
assigning  ratings  is  that  contractors  will  not  be 
assessed  a  rating  lower  than  Satisfactory  solely  for 
not  performing  beyond  the  requirements  of  the 
contract 

Marginal 

Performance  docs  not  meet  some 
contractual  requirements.  The  contractual 
performance  of  tlx:  element  or  sub- 
element  being  assessed  reflects  a  serious 
problem  for  which  the  contractor  has  not 
yet  identified  corrective  actions.  The 
contractor’s  proposed  actions  appear  only 
marginally  effective  or  were  not  fully 
implemented. 

To  justify  Marginal  performance,  identify  a 
significant  event  in  each  category  that  the  contractor 
had  trouble  overcoming  and  state  bow  it  impacted 
the  Government  A  Marginal  rating  should  be 
supported  by  referencing  the  management  tool  that 
notified  the  contractor  of  the  contractual  deficiency 
(c.g..  management  quality,  safety,  or  environmental 
deficiency  report  or  letter). 

Unsatisfactory 

Performance  docs  not  meet  most 
contractual  requirements  and  recovery  is 
not  likely  in  a  timely  manner.  The 
contractual  performance  of  tlx:  element  or 
sub-clcmcM  contains  a  serious  problcm(s) 
for  which  the  contractor's  corrective 
actions  appear  or  were  ineffective. 

To  justify  an  Unsatisfactory  rating,  identify  multiple 
significant  events  in  each  category  that  the  contractor 
had  trouble  overcoming  and  state  how  it  impacted 
the  Government  A  singular  problem,  however, 
could  be  of  such  serious  magnitude  that  it  alone 
constitutes  an  unsatisfactory  rating.  An 

Unsatisfactory  rating  should  be  supported  by 

referencing  the  management  tools  used  to  notify  the 
contractor  of  the  contractual  deficiencies  (c.g., 
management,  quality,  safety,  or  environmental 
deficiency  reports,  or  letters). 

NOTH  1 :  Plus  or  minus  signs  may  be  used  to  indicate  an  improving  (+)  or  worsening  (-)  trend  insufficient  to  change  the 
assessment  status. 

NOTE  2:  N.'A  (not  applicable)  should  be  used  if  the  ratings  arc  not  going  to  be  applied  to  a  particular  area  for  evaluation. 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 


■85- 


_^J<atin"> 

Exceptional 


Very  Good 


Satisfactory 


^j^iIuatioi^LatinjjjOJcfinitionj^^ilizatioiMjnimainJusincss^ 


Definition 


Note 


Exceeded  all  negotiated  subcontracting 
goals  or  exceeded  at  least  one  goal  and 
met  all  of  the  other  negotiated 
subcontracting  goals  for  the  current 
period.  Had  exceptional  success  with 
initiatives  to  assist,  promote,  and  utilize 
small  business  (SB),  small  disadvantaged 
business  (SDB),  wo  men -owned  small 
business  (WOSB).  HUBZonc  small 
business,  veteran-owned  small  business 
(VOSB)  and  service  disabled  veteran 
owned  snail  business  (SDVOSB). 
Complied  with  FAR  52.219-8,  Utilization 
of  Small  Business  Concents.  Exceeded 
any  other  small  business  participation 
requirements  incorporated  in  the  contract, 
including  the  use  of  small  businesses  in 
mission  critical  aspects  of  the  program. 
Went  above  and  beyond  the  required 
elements  of  the  subcontracting  plan  and 
other  snull  business  requirements  of  the 
contract.  Completed  and  submitted 
Individual  Subcontract  Reports  and  or 
Summary  Subcontract  Reports  in  an 
accurate  and  timely  manner. _ 

Met  all  of  the  negotiated  subcontracting 
goals  in  the  traditional  socio-economic 
categories  (SB.  SDB  and  WOSB)  and  met 
at  least  one  of  the  other  socio-economic 
goals  (HUBZonc,  VOSB,  SDVOSB)  for 
the  current  period.  Had  significant 
success  with  initiatives  to  assist,  promote 
and  utilize  SB.  SDB.  WOSB.  HUBZonc, 
VOSB,  and  SDVOSB.  Complied  with 
FAR  52.2 19-8.  Utilization  of  Small 
Business  Concerns.  Met  or  exceeded  any 
other  small  business  participation 
requirements  incorporated  in  the  contract, 
including  the  use  of  small  businesses  in 
mission  critical  aspects  of  the  program. 
Endeavored  to  go  above  and  beyond  the 
required  elements  of  the  subcontracting 
plan.  Completed  and  submitted 
Individual  Subcontract  Reports  andor 
Summary  Subcontract  Reports  in  an 
accurate  and  timely  manner. _ 

Demonstrated  a  good  faith  effort  to  meet 
all  of  the  negotiated  subcontracting  goals 
in  the  various  socio-economic  categories 
for  the  current  period.  Complied  with 
FAR  52.219-8.  Utilization  of  Small 
Business  Concerns.  Met  any  other  small 
business  participation  requirements 


To  justify  an  Exceptional  rating,  identify'  multiple 
significant  events  and  state  how  they  were  a  benefit 
to  small  business  utilization.  A  singular  benefit, 
however,  could  be  of  such  magnitude  that  it 
constitutes  an  Exceptional  rating.  Ensure  that  small 
businesses  arc  given  meaningful,  innovative  work 
directly  related  to  the  project  rather  than  peripheral 
work,  such  as  cleaning  offices,  supplies,  landscaping, 
etc.  Also,  there  should  have  been  no  significant 
weaknesses  identified. 


To  justify  a  Very  Good  rating,  identify'  a  significant 
event  and  state  how  they  were  a  benefit  to  small 
business  utilization.  Ensure  that  small  businesses  are 
given  meaningful,  innovative  work  directly  related  to 
the  prefect  rather  than  peripheral  work,  such  as 
cleaning  offices,  supplies,  landscaping,  etc.  There 
should  be  no  significant  weaknesses  identified. 


To  justify  a  Satisfactory  rating,  there  should  have 
been  only  minor  problems,  or  major  problems  the 
contractor  has  addressed  or  taken  corrective  action. 
Thcrc  should  have  been  no  significant  weaknesses 
identified.  A  fundamental  principle  of  assigning 
ratings  is  that  contractors  will  not  be  assessed  a 
ratine  lower  than  Satisfactory  solely  for  not 
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Marginal 


Unsatisfactory 


included  in  the  contract.  Fulfilled  tbc 
requirements  of  the  subcontracting  plan 
included  in  the  contract.  Completed  and 
submitted  Individual  Subcontract  Reports 
andor  Summary  Subcontract  Reports  in 
an  accurate  and  timely  manner. _ 

Deficient  in  meeting  key  subcontracting 
plan  elements.  Deficient  in  complying 
w  ith  FAR  52.219-8.  Utilization  of  Small 
Business  Concerns,  and  any  other  small 
business  participation  requirements  in  the 
contract.  Did  not  submit  Individual 
Subcontract  Reports  and/or  Summary 
Subcontract  Reports  in  an  accurate  or 
timely  manner.  Failed  to  satisfy  one  or 
more  requirements  of  a  corrective  action 
plan  currently  in  place,  however,  docs 
show  an  interest  in  bringing  performance 
to  a  satisfactory  level  and  has 
demonstrated  a  commitment  to  apply  the 
necessary  resources  to  do  so.  Required  a 
corrective  action  plan. _ 

Noncompliant  with  FAR  52.219-8  and 
52 .2 19-9,  and  any  other  small  business 
participation  requirements  in  tbc  contract. 
Did  not  submit  Individual  Subcontract 
Reports  and'or  Summary  Subcontract 
Reports  in  an  accurate  or  timely  manner. 
Showed  little  interest  in  bringing 
performance  to  a  satisfactory  level  or  is 
generally  uncooperative.  Required  a 
corrective  action  plan. 


performing  beyond  the  requirements  of  the  contract 


To  justify  Marginal  performance,  identify’  a 
significant  event  that  the  contractor  had  trouble 
overcoming  and  how  it  impacted  small  business 
utilization.  A  Marginal  rating  should  be  supported 
by  referencing  the  actions  taken  by  the  government 
that  notified  the  contractor  of  the  contractual 
deficiency. 


To  justify  an  Unsatisfactory  rating,  identify  multiple 
significant  events  that  cite  contractor  had  trouble 
overcoming  and  state  how  it  impacted  small  business 
utilization.  A  singular  problem,  however,  could  be 
of  such  serious  magnitude  that  it  alone  constitutes  an 
Unsatisfactory  rating.  An  Unsatisfactory  rating 
should  be  supported  by  referencing  the  actions  taken 
by  tlx  government  to  notify  the  contractor  of  the 
deficiencies.  When  an  Unsatisfactory  rating  is 
j  ustified,  the  contracting  officer  must  consider 
whether  the  contractor  made  a  good  faith  effort  to 
comply  with  the  requirements  of  the  subcontracting 
plan  required  by  FAR  52.2 1 9-9  and  follow  the 
procedures  outlined  in  FAR  52.219-16.  Liquidated 
Damagcs-Subcontracting  Plan. 


NOTH  1 :  Plus  or  minus  signs  may  be  used  to  indicate  an  improving  (+)  or  worsening  (-1  trend  insufficient  to  change  assessment 
status. 

NOTH  2:  For  subcontracting  plans  under  the  DoD  Comprehensive  Small  Business  Subcontracting  Plan  (Test  Program), 

DFARS  252.2 19-7004  (deviation),  the  ratings  entered  in  CPARS  shall  mirror  those  assigned  by  the  Defense  Contract 
Management  Agency  who  is  responsible  for  monitoring  such  plans. 

NOTE  3:  Generally,  zero  percent  is  not  a  goal  unless  the  Contracting  Officer  determined  when  negotiating  tlx 
subcontracting  plan  that  no  subcontracting  opportunities  exist  in  a  particular  socio-economic  category.  In  such  eases,  tlx 
contractor  shall  be  considered  to  have  met  the  goal  for  any  socio-economic  category  where  tbc  goal  negotiated  in  the  plan 
was  zero. 
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